Server Quality Checklist

Tool Scores

• get_frontmatterB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns key-value metadata but does not describe error handling (e.g., for missing files or malformed YAML), performance traits, or side effects, which are critical for a file-parsing operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and outcome. Every word earns its place, with no redundant information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that parses files. It does not cover error cases, return format details (e.g., structure of key-value pairs), or operational constraints, leaving significant gaps in understanding how to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'file' documented as 'Path to the markdown file'. The description adds no additional parameter semantics beyond this, such as format requirements or examples, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Parse YAML frontmatter'), the resource ('at the start of a markdown file'), and the scope ('between --- delimiters'), distinguishing it from siblings like find_code_blocks or get_section by focusing on metadata extraction rather than content retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites (e.g., file must exist), exclusions (e.g., files without frontmatter), or comparisons to siblings like list_files or search_docs, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_code_blocksB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'find all' and optional filtering, but doesn't disclose behavioral traits like what happens if the file doesn't exist, how errors are handled, whether it's read-only (implied but not stated), or the format of returned results. For a tool with no annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste: the first states the core purpose, the second adds the optional filter detail. It's appropriately sized and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters with full schema coverage and no output schema, the description is adequate but incomplete. It covers the basic purpose and optional filtering, but lacks details on error handling, result format, or behavioral constraints that would be helpful for an agent. No annotations exist to compensate, making this minimally viable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (file path and optional language filter). The description adds minimal value by restating the optional language filter with examples, but doesn't provide additional semantics beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'find' and resource 'fenced code blocks in a markdown file', with specific mention of filtering by language. It distinguishes from siblings like get_frontmatter or list_headings by focusing on code blocks, but doesn't explicitly contrast with search_docs which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for markdown files with code blocks and mentions optional language filtering, but doesn't explicitly state when to use this tool versus alternatives like search_docs or get_section. No guidance on prerequisites or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_headingsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral context. It states what the tool does but doesn't disclose how it behaves—such as error handling for invalid files, output format details, or whether it reads files safely. This leaves significant gaps for a tool that interacts with file systems.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose without unnecessary words. It is front-loaded with the core action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no annotations, no output schema), the description is adequate but incomplete. It specifies the input type (markdown file) and output concept (table of contents) but lacks details on output format or error conditions, which are important for a file-reading tool with no structured output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the single 'file' parameter as a path. The description adds no additional meaning beyond implying markdown files, which is redundant with the tool's stated purpose. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all headings') and resource ('in a markdown file') with precise scope ('# through ###### as a table of contents'). It distinguishes itself from sibling tools like 'get_section' or 'search_docs' by focusing exclusively on heading extraction rather than broader document operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for markdown files needing a table of contents, but provides no explicit guidance on when to use this tool versus alternatives like 'get_section' for specific content or 'search_docs' for broader searches. It lacks any mention of prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sectionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: the extraction logic (content from heading until next same/higher-level heading) and case-insensitive matching (implied from schema). However, it lacks details on error handling (e.g., if heading not found), performance, or output format beyond content extraction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste, front-loading the core purpose and following with precise behavioral details. Every word earns its place, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (extraction based on heading levels) and no annotations or output schema, the description is mostly complete. It explains what the tool does and how it behaves, but lacks details on return values (e.g., structured vs. raw text) and error scenarios, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (file path and heading text). The description adds no additional parameter semantics beyond what the schema provides, such as format examples or edge cases. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Extract a section by heading'), resource ('from a markdown file'), and scope ('Returns content from that heading until the next heading of same or higher level'). It distinguishes itself from siblings like list_headings (which lists headings) and find_code_blocks (which finds code blocks).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (extracting content from markdown files based on headings) but doesn't explicitly state when to use this tool versus alternatives like list_headings or search_docs. No guidance is provided on prerequisites, exclusions, or specific scenarios where this tool is preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adequately describes the recursive scanning behavior and output sorting, but doesn't mention potential limitations like permission requirements, error handling, performance characteristics, or what happens with empty directories. The description doesn't contradict any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place: the first specifies what the tool does, and the second describes the return format. There's zero wasted language and it's front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one parameter and no output schema, the description provides sufficient context about what it does and returns. It could be more complete by mentioning error cases or performance considerations, but covers the essential functionality well given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the baseline is 3. The description adds value by specifying that the tool lists '.md files' and works 'recursively', which provides context beyond the schema's parameter documentation. However, it doesn't elaborate on the directory parameter format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all .md files'), resource (files in a directory), scope (recursively), and output format (relative paths sorted alphabetically). It distinguishes itself from sibling tools like 'search_docs' by focusing on listing rather than searching or extracting content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving .md files from a directory structure, but provides no explicit guidance on when to use this tool versus alternatives like 'search_docs' or 'find_code_blocks'. It doesn't mention prerequisites, exclusions, or comparative scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the search scope (full-text across .md files), output format, and a key limitation (50 result limit). However, it doesn't mention other behavioral aspects like performance characteristics, error handling, or whether the search is recursive through subdirectories. The description doesn't contradict any annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and well-structured in just two sentences. The first sentence establishes the core functionality and scope, while the second sentence adds crucial behavioral constraints (output format and result limit). Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search operation with 2 parameters), no annotations, and no output schema, the description does a good job covering the essentials: purpose, scope, output format, and a key limitation. However, it could be more complete by addressing whether the search is recursive, how results are sorted, or what happens with empty results. The absence of an output schema means the description must fully explain return values, which it partially does but not comprehensively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, providing clear documentation for both parameters. The description doesn't add any parameter-specific information beyond what's in the schema (e.g., it doesn't explain directory path formats or query syntax further). According to the scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no additional param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('full-text search'), target resource ('.md files in a directory'), and output format ('returns file, line number, and matching line'). It distinguishes itself from sibling tools like list_files (which lists files without searching content) and get_section (which retrieves specific sections rather than searching across files).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool: for searching content within .md files. However, it doesn't explicitly state when NOT to use it or mention alternatives among the sibling tools (e.g., using list_files to browse files without searching content, or find_code_blocks for specific code block searches). The 'limited to 50 results' constraint offers some usage guidance but doesn't fully address alternative scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.