Server Quality Checklist

Tool Scores

• local_askA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must cover behavioral traits. It only states 'no file reading' and hints at local model usage. No mention of safety, side effects, rate limits, or auth. For a mutation-like tool (query), more transparency is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, no extraneous words. Efficient and clear about the tool's raison d'être.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters, no output schema, and no annotations, the description is too brief. It lacks details on return format, error handling, token management, and the difference between 'code' and 'light' models.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds no parameter-specific semantics beyond the schema—e.g., it doesn't explain the 'model' enum values or 'max_tokens' default implications.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool poses a free-form question to the local model without file reading, and lists specific use cases (boilerplate, reformulation, translation, commit message, regex). This differentiates it from siblings like local_map, local_digest, which likely involve file processing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides explicit use cases and notes it's for tasks not worthy of the cloud model. It implies file-reading tasks use siblings, but does not explicitly name alternatives or give when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• local_mapA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers behavior: it states the operation is sequential, not parallelizable by GPU, and estimates seconds per file. It also mentions returning a result per file. This is good transparency, though it doesn't detail side effects or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively compact, with three sentences: main purpose, examples, and performance note. It starts with the core function, so it's well front-loaded. Could be slightly more concise, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters and no output schema, the description covers the core purpose and performance but does not specify the output format or structure (e.g., how results per file are returned). It is adequate for basic use but incomplete for an agent needing to parse results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description does not need to add much. It repeats the 'instruction' purpose from the schema but adds no new meaning. The description does not elaborate on patterns, cwd, model, max_files, or max_tokens beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that local_map applies the same instruction to each file separately and returns one result per file. It provides specific examples like classifying files, extracting a field, and detecting patterns, which helps distinguish it from siblings like local_digest, local_ask, and local_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear use cases (batch processing, classification, extraction) and performance characteristics (sequential, seconds per file). However, it does not explicitly state when not to use this tool or compare it to alternatives, so it lacks exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• local_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, but the description adequately describes the behavioral traits: it is a read-only status check that returns model availability, loaded model, and context. It implies no side effects, which is appropriate for a simple status tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the purpose and usage. No wasted words; front-loaded with the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description provides enough context about what it returns (available models, loaded model, configured context). It is complete enough for a simple status check, though a native English speaker might need translation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so schema coverage is 100%. The description does not add parameter details, but none are needed. Baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the state of LM Studio, listing available models, loaded model, and configured context. It includes a specific usage scenario (error or unexpected slowness), distinguishing it from siblings like local_digest, local_map, and local_ask.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to call the tool (in case of error or slowness). It does not mention when not to use it, but the context is clear given the sibling tools have different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• local_digestA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that raw file content never enters Claude's context (token economy), that only the result is returned, and that map-reduce is automatic for large volumes. These are significant behavioral traits. Missing details like authorization or error handling are acceptable for a local read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each essential. First sentence states core functionality, second highlights the primary benefit (token savings), third gives usage guidance and map-reduce note. No fluff, front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, all covered by schema with additional description, no output schema needed (description states only result returned). The description fully explains how the tool works, when to use it, and its token-saving behavior. An agent can correctly select and invoke this tool based solely on this definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaningful detail: 'model' enum values are explained with usage context (default 'code' is recommended everywhere, 'light' needs larger max_tokens), 'patterns' are clarified as globs relative to cwd, and 'instruction' is exemplified. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Lit' - reads) and resource ('fichiers EN LOCAL') and clearly states the action: apply an instruction to the content and return only the result. It distinguishes from manual reading by highlighting token savings and map-reduce behavior, and implicitly differentiates from siblings like local_map, local_ask, and local_status by focusing on digestion with an instruction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends using this tool before reading a large file yourself ('A privilegier avant de lire soi-meme un gros fichier ou un ensemble de fichiers'). It also explains the map-reduce fallback for large volumes. However, it does not explicitly state when not to use it or provide direct comparisons to sibling tools beyond the implied purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.