Server Quality Checklist

Tool Scores

• lc_outlinesC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool 'Returns excerpted content highlighting important sections,' which implies a read-only operation, but doesn't specify whether it modifies files, requires specific permissions, has rate limits, or what the output format entails. The description lacks details on behavioral traits like error handling or performance characteristics, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively concise with two sentences and a parameter list, but it's not optimally structured. The first sentence clearly states the purpose, but the parameter list includes undocumented items that conflict with the schema, adding noise. While it avoids excessive verbosity, the inconsistency reduces its effectiveness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (involving file processing and rule-based selection) and the presence of an output schema, the description is incomplete. It doesn't explain what 'excerpted content' or 'important sections' mean, what file types are supported, or how rules are applied. The parameter discrepancy further undermines completeness. Although the output schema might cover return values, the description lacks essential context for proper tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description lists three parameters (root_path, rule_name, timestamp), but the input schema only documents one parameter (root_path) with 0% schema description coverage. This creates a contradiction and leaves two parameters (rule_name, timestamp) entirely undocumented in both the schema and description. The description fails to add meaningful semantics beyond the schema and doesn't compensate for the low coverage, resulting in confusion about parameter validity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns excerpted content highlighting important sections in all supported files.' It specifies the verb ('Returns excerpted content') and resource ('all supported files'), and distinguishes it from siblings by focusing on content highlighting rather than change detection, missing files, or rule instructions. However, it doesn't explicitly differentiate from siblings in the description text itself, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (lc_changed, lc_missing, lc_rule_instructions). It lists parameters but doesn't explain the context or prerequisites for invoking the tool, such as what types of files are supported or when excerpting is appropriate. This leaves the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lc_rule_instructionsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool 'provides' instructions, implying a read-only operation, but doesn't clarify if this requires authentication, has rate limits, affects system state, or what format the instructions take. The description is too vague about behavioral traits beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences: one stating the purpose and another documenting the parameter. It's front-loaded with the main purpose. However, the parameter documentation could be integrated more smoothly rather than as a separate 'Args:' section.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (which handles return values), one parameter, and no annotations, the description is minimally adequate. It states the purpose and documents the parameter, but lacks behavioral context and usage guidance. For a tool with output schema support, this is borderline complete but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It documents the single parameter 'root_path' with minimal context: 'Root directory path.' This adds some meaning beyond the schema's type information but doesn't explain what this path represents, format expectations, or why it's required. With one parameter and low coverage, this is insufficient compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Provides step-by-step instructions for creating custom rules.' This is a specific verb ('provides') + resource ('instructions') combination that explains what the tool does. However, it doesn't differentiate from sibling tools like lc_changed, lc_missing, or lc_outlines, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of context, prerequisites, or comparisons with sibling tools. The only usage hint is the parameter documentation, which doesn't address tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lc_missingC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'retrieves' missing context, implying a read-only operation, but doesn't address permissions, rate limits, error handling, or what 'retrieving' entails operationally. The description lacks behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a purpose statement followed by parameter explanations in a list format. It's front-loaded with the main function and avoids unnecessary details, though the parameter explanations could be more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters with 0% schema coverage and no annotations, the description provides basic context but lacks depth. An output schema exists, so return values needn't be explained, but the tool's operational behavior and parameter semantics remain under-specified for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides brief explanations for each parameter (e.g., 'Root directory path', 'Type of data'), but these are minimal and don't fully clarify usage. For example, 'data' is described as 'JSON string containing the data' without detailing structure or examples beyond file paths. The description adds some value but doesn't fully compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'retrieving missing context' with specific resource types (files, implementations, excluded sections). It uses the verb 'retrieving' and identifies the resource scope, though it doesn't explicitly differentiate from sibling tools like lc_changed or lc_outlines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description mentions the tool is 'unified' but doesn't specify use cases, prerequisites, or exclusions compared to sibling tools like lc_changed or lc_outlines.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lc_changedB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does (returns a list of modified files) but doesn't describe important behavioral aspects like: what format the returned list is in, whether it's recursive or only checks the root directory, error handling for invalid paths, permissions required, or rate limits. The description is minimal and lacks behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise and well-structured. It starts with the core purpose, then lists parameters with brief explanations. Every sentence earns its place, and there's no unnecessary verbiage. The example path format adds clarity without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there's an output schema (which should describe the return format), the description doesn't need to explain return values. However, for a file system tool with no annotations, it should provide more behavioral context about traversal depth, error conditions, or permissions. The description covers the basic purpose and parameters adequately but could be more complete about operational behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides clear semantic meaning for both parameters beyond what the schema offers. The schema has 0% description coverage (just titles 'Root Path' and 'Timestamp'), but the description explains: 'root_path: Root directory path (e.g. '/home/user/projects/myproject')' and 'timestamp: Unix timestamp to check modifications since'. This adds valuable context about format and purpose that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns list of files modified since given timestamp.' This is a specific verb ('returns list of files') with a clear resource scope ('modified since given timestamp'). It doesn't explicitly differentiate from sibling tools like 'lc_missing' or 'lc_outlines', but the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or any context for choosing this over other options. The only usage context is implied by the parameters, but there's no explicit when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.