Server Quality Checklist

Tool Scores

• get_job_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It only says 'check the status' without noting whether the job may remain queued, whether the operation is idempotent and safe to pole, whether it can return errors after the job has failed, or whether results are singal-shot or persistent. For an async polling tool, this lacks meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One short sentence conveys the core purpose with zero filler. The phrasing is front-loaded and readable. However, it is terse to the point of omitting useful context, so it loses a point for under-specification rather than for waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although the output schema exists so return values need not be described, there is no guidance on when to call this tool, where job_id comes from, or what behavioral expectations exist. Given the async nature and the presense of sibling tools like get_index_status and index_document, this description is incomplete for reliable tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the tool description must compensate by explaining job_id. It does not. The parameter name alone is fairly self-evidt but the description does not clarify that job_id is the identifier returned by index_document or index_directory when an async job is created, nor does it mention any format or origin. This leaves the agent guessing where to obtain the ID.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Check') and a precise resource ('status of an async indexing job'). The 'async indexing job' qualifier differentiates this from the sibling get_index_status, which likely targets the overall index rather than an individual job. The purpose is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like index_document, index_directory, or get_index_status. There is no mention that this should be called with a job_id returned by an asynchronous indexing submision, and no exclusions or alternative routing are provided. The agent is left to infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aggregateB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It communicates the core behavior of read-only aggregation and mentions common operations like count/sum/group-by, but it does not disclose limitations, edge cases, side effects, or behavior around filter and agg_column.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded, with the core purpose in the first sentence and a useful example following immediately. Every sentence earns its place, and there is no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema provides return-type context, and the example supports basic usage, but the tool is not fully described for all five parameters. Missing semantics for filter and agg_column leave meaningful gaps for an agent trying to use sum or filtered aggregations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. The example clarifies table, group_by, and agg, but filter and agg_column are left unexplained, including the crucial relationship between agg_column and agg for sum queries. This is only partial compensation for undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs simplified aggregation on a DuckDB table, with a concrete example showing count/group-by usage. It is specific about the verb and resource, but it does not explicitly distinguish itself from sibling tools such as query_tables beyond the vague 'Easier than SQL' comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Easier than SQL for common count/sum/group-by queries' implies when to use the tool, but it does not name alternatives or state when not to use it. Usage guidance is present but only implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_index_statusA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It conveys a read-only status operation through 'Return', but it does not mention side effects, access requirements, or any potential performance considerations. This is adequate but not especially rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tightly worded sentance that front-loads the verb and resource. Every phrase adds meaningful information with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless status tool with an output schema available, the description supplies the key return categories (health, counts, roots, dependency versions) and scopes it to Qdrant/DuckDB. The main gaps are the unexplained 'roots' term and lack of differentiation from sibling health tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema has nothing to document. The description compensates by indicating what kind of information is returned, which is all an agent needs for parameterless invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and concrete subject matter (Qdrant/DuckDB health, counts, roots, dependency versions). This clearly identifies the tool's purpose, though 'roots' is somewhat ambiguous and there is no direct differentiation from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives such as 'health' or 'get_job_status'. The agent is left to infer the appropriate use case from the tool name and brief description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_directoryA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It does reveal meaningful behavior: diff reporting, force re-indexing behavior, and the timeout risk addressed by async_index. However, it does not mention side effects on the existing index, permissions needed, or whether indexing is safe/reversible, leaving the operational impact partially undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. The first sentence states the core purpose, the second describes return categories, and the next two sentences flag the most important optional behaviors. Every sentence provides useful information with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return values do not need full description, and the description does summarize output categories. But with no annotations and no schema-level parameter docs, the description leaves key call semantics unexplained, especially limit, pattern, recursive, and how dir defaults to '.'. For a six-parameter tool, this is not complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It only explains force and async_index; dir, limit, pattern, and recursive are left to bare titles and defaults. This is insufficient for a tool with six parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Index supported files in a directory with diff reporting.' It also clarifies what the operation returns (new/changed/unchanged/failed files), which makes the tool's purpose concrete and distinct from sibling index_document by directory vs document scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives useful conditional guidance for force and async_index, but does not say when to choose this tool over alternatives like index_document, search_documents, or get_index_status. Usage context is mostly implied by the tool's purpose rather than explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentsA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It meaningfully explains hybrid retrieval, optional metadata filtering, structured citations, and cross-encoder reranking, which helps an agent anticipate results and configuration effects. It does not mention auth, rate limits, or whether results are paginated, but the disclosed retrieval behavior is substantive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three focused sentences with the primary action front-loaded. The filter example and reranking note each add value without redundant filler, making it an efficient and well-structured definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives enough for a basic search call and mentions result citation structure, and an output schema is presumably available to define the return shape. However, the missing semantics for limit and document_id, plus the absence of explicit differentiation from query_tables, leave noticeable gaps for correct tool selection and full argument usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only addresses filter via an example and implicitly covers query. The limit and document_id parameters are not explained, leaving a significant semantic gap for agents that need to constrain result count or scope to a specific document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the tool's action 'Search indexed documents' and adds a distinctive retrieval approach ('hybrid dense+sparse retrieval'), so the core purpose is clear. However, it does not explicitly differentiate this from sibling tools like query_tables, leaving some ambiguity about which search-like tool is appropriate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching indexed documents and shows an example filter key, but it gives no explicit guidance about when to prefer this tool over query_tables or when the alternative should be used. No exclusions or when-not-to-use conditions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It indicates a read operation via 'Return' but does not discuss error behavior, authentication requirements, or the potentially large payload implied by 'all indexed chunks'. This is adequate for a simple read but not richly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It states the action, the resource, and the scope in order of importance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return value details are covered elsewhere. However, the description does not mention how document_id should be obtained, what happens when the id is invalid or not yet indexed, or whether the response is large. For a one-parameter read it is nearly complete but leaves small, relevant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must add parameter meaning. It does clarify that document_id selects the document whose chunks and provenance are returned, but it does not specify the expected id format or how to obtain a valid id. This is minimal but useful context beyond the schema's type and title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Return all indexed chunks and provenance for one document id.' It specifies a single document id, which distinguishes this get-by-id tool from siblings like list_documents and search_documents without requiring the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: call this when you have a specific document id and need its chunks/provenance. It does not mention alternatives or when not to use it, so usage guidance is only implied, not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_documentsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the burden of behavioral disclosure. It conveys that this is a read-only listing operation and shows what data is returned, but it does not mention pagination, result size limits, ordering, or whether all indexed files are always returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one well-structured sentence with no filler. The verb and resource are front-loaded, and every listed field adds useful information about the output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, this is largely complete: it identifies the resource and the key returned fields. The only notable gap is the absence of pagination or scale behavior, but since the tool takes no inputs and has an output schema, the description is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty, so there is nothing for the description to elaborate. The baseline of 4 applies because no parameter semantics are needed; the description appropriately focuses on the output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('indexed files'), and names the returned metadata fields (hashes, chunk counts, timestamps, source paths). This clearly distinguishes it from sibling tools like search_documents and get_document, which imply filtering or retrieving individual documents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended usage is implied by the word 'List' and the named fields, but the description does not explicitly say when to use this tool instead of search_documents or get_document. It also does not mention any alternatives or conditions that would exclude its use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• healthA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly signals a read-only operation, which is valuable safety context. However, it does not describe what 'health' covers, whether it contacts live backends, or how results are returned; the output schema partially compensates.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no wasted words. The most important information — that this is a read-only health snapshot covering the MCP and its backends — is front-loaded and immediately actionable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter health check, the description is sufficiently complete. It states the tool's purpose, scope, and read-only nature. The presence of an output schema covers return value details, so nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is already complete. The description does not need to explain parameter meaning, and the baseline of 4 applies. No parameter semantics are missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and resource ('read-only health snapshot for this MCP and its backends'). It clearly distinguishes this tool from siblings like get_index_status or get_job_status by framing it as a general health check across the MCP and its backends.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The read-only health snapshot phrasing implies it is for checking system health, but the description does not explicitly state when to use this tool versus the alternative status/config tools. No exclusions or alternative routing are provided, so usage guidance is mostly implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_configA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It explicitly discloses read-only behavior and the absence of secrets, which is valuable safety-relevant information. It does not discuss authentication or error behavior, but this is minor for a simple, parameterless reader.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One tightly worded sentence carries the full semantic load: read-only, safe, runtime configuration, and no secrets. It is front-loaded and contains no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters, no annotations, and an output schema present, this description is nearly complete for the agent's needs. It conveys what the tool returns and what it explicitly excludes. Slightly more context about the source or scope of the configuration could push it to a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema description coverage is 100%, so there is nothing the description needs to explain about arguments. The baseline for parameterless tools is 4, and the description satisfies that without adding unnecessary parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and resource ('runtime configuration'), and adds meaningful scope ('without secrets'). It clearly distinguishes itself from sibling tools focused on documents, tables, indexes, and status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'runtime configuration' implies when the tool should be used, but the description does not explicitly state when not to use it or name alternatives. An agent must infer how this differs from get_document or get_index_status rather than being told.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_tablesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It communicates that this is a read-style SQL query via supported operations like COUNT, GROUP BY, JOIN, and WHERE, which implies no data mutation. However, it does not explicitly state read-only behavior, result limits, or error/edge-case behavior, leaving some ambiguity for arbitrary SQL.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, each earning its place: the core action, the prerequisite discovery step, and the supported SQL feature set. The most important usage instruction is front-loaded after the one-line purpose, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter and an output schema exists, the description covers the essential call context: what the SQL targets, how to find valid table names and schemas, and which operations are supported. It misses minor items like explicit read-only confirmation, but for a simple query tool with low parameter complexity it is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only the parameter name 'sql' with no description, so the description must compensate. It does so by clarifying that the parameter is a SQL query against DuckDB, listing supported clauses, and linking to list_tables for schema discovery. This is sufficient for the single parameter, though a small syntax example would make it even stronger.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Execute a SQL query on extracted tables stored in DuckDB.' It goes beyond a bare name by specifying the database engine and the supported SQL operations, making it clearly distinct from sibling tools like list_tables or aggregate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs the agent to 'Use list_tables() first to discover available table names and column schemas,' which gives clear before-use context. It does not explicitly mention when not to use this tool or name an alternative for non-query needs, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden, and it does a good job for a list tool: 'List all extracted tables' clearly indicates a read-only operation and defines the scope, while the return-field list sets expectations. It does not mention potential performance characteristics or edge cases, but for a simple list operation this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The primary action and scope are front-loaded, followed by the list of returned data, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with an output schema available, the description is complete. It states exactly what is listed and what information is returned, giving an agent all it needs to decide to call the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the baseline is 4. There are no parameter details to document, and the description appropriately focuses on the operation and return values rather than nonexistent arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description has a specific verb ('List'), a clear resource ('all extracted tables in the DuckDB relational store'), and enumerates the returned fields (table names, source documents, column schemas, row counts). This clearly distinguishes it from sibling tool list_documents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The action and scope imply usage for discovering available tables, but the description does not explicitly compare it with alternatives like query_tables or list_documents, nor state when not to use it. There is no explicit guidance on selection among siblings, so it earns an 'implied usage' score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_documentA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it does well: it discloses that content hash is checked, that unchanged documents are skipped, that force overrides this, and that async_index shifts to background and returns a job_id. It aids the agent in predicting side effects and return behavior beyond the input schema. It stops short of a 5 because it doesn't mention error/validation-response specifics or permission prerequisites, though the output schema likely covers return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the main purpose, and each line adds concrete value: path resolution, supported formats, return behavior, force, and async options. There is no redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple 3-parameter tool with an output schema and no nested objects, the description covers all relevant operational concerns: supported file types, path context, hash-change optimization, re-index override, and background execution mode. The agent has enough to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does: it explains path semantics (relative vs absolute), force behavior (re-index despite unchanged hash), and async_index behavior (background execution, returns job_id). This gives the agent meaningful guidance that the raw schema field names alone would not provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Convert and index one document', which clearly differentiates this from sibling tools like index_directory (one vs. directory) and search_documents (indexing vs searching). The mention of 'one document' plus the supported format list fully establishes its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states that the tool handles a single document from a configured root, which implicitly tells the agent to choose index_directory for batch indexing. It also gives explicit guidance on when to use force=True and async_index=True, and describes the relative/absolute path options. It lacks an explicit when-not-to-use statement naming alternatives, which keeps it just shy of 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.