Server Quality Checklist

Tool Scores

• rtfm_booksB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full behavioral burden. It does convey the read-only 'List' behavior and the presence of pagination, which is useful context. But it does not explain what the per-corpus summary contains, what the response looks like, or whether cross-project access (via the project parameter) has any caveats.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler or repetition of schema details. Every word adds value: the operation, the per-corpus summary, and pagination.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With four parameters, no output schema, and no annotations, the description is too thin to fully specify the returned shape of the per-corpus summary or the default/current project behavior. It covers the high-level operation but leaves an agent guessing about response format and cross-project semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter is already documented in the input schema. The description only vaguely relates to corpus and pagination without adding meaning beyond the schema, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific operation ('List indexed books') and adds distinctive qualifiers (per-corpus summary, pagination) that separate it from the search, coverage, and history siblings. However, it never explicitly names a sibling or defines the per-corpus summary, so it is clear but not fully differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to choose this tool over rtfm_search, rtfm_discover, or rtfm_coverage, and no exclusions are given. The only signal is the literal purpose, which is not enough to steer an agent among eight siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rtfm_graphB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It discloses that the tool returns graph neighbors and hints at scoping via relation types, which is useful. However, it does not clarify whether traversal is immediate only, whether results are ordered or limited, whether filtering is optional, or any side effects (though this appears read-only). The lack of annotation support leaves room for ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero filler. Every word contributes: the verb, the resource, the graph-neighbor scope, and the enumerated relation types. This is model conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-style tool with 100% parameter schema coverage, the description covers the core purpose but leaves gaps: no default behavior for 'direction' or 'relation_type' is stated, no output format is described, and no distinction from sibling tools is provided. Since there is no output schema, a bit more detail about the result shape or default behavior would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all three parameters. The description adds a small amount of context by listing relation types in prose, but it does not meaningfully enrich the semantic understanding beyond what the schema provides. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Show'), a resource ('graph neighbors of a file'), and enumerates the relation types (imports, links, includes, citations). It clearly communicates what the tool does, though it doesn't explicitly differentiate from the sibling tool rtfm_expand, which may have a similar graph-expansion role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like rtfm_expand or rtfm_context. The description implies use when needing neighbor information, but there is no explicit context, prerequisites, or exclusions to help an agent choose among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rtfm_discoverA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It adds a useful performance trait ('~1 second') and implies a read-only scan, but it does not disclose behaviors like recursive traversal, handling of invalid paths, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence that communicates the action, expected output, and rough execution time without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description adequately explains what is returned and the expected scope. Some guidance about errors or edge cases would improve completeness, but it is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the only parameter, 'path', is already described as 'project root directory (default ".")'. The description adds little semantic detail beyond the schema; it merely repeats the notion of a project directory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Scan a project directory' and return a 'structural map' with concrete output elements. It is clear but does not explicitly differentiate itself from sibling tools like rtfm_search or rtfm_context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: if you need a structural overview of a project directory, this is the tool. However, there is no explicit guidance about when to prefer it over sibling tools or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rtfm_coverageA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and does well: it clarifies that the denominator is the scan's file list, not the whole directory, and that excluded files are not counted as gaps. It does not disclose the return format, but the core measurement semantics are transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short sentences, each adding a distinct piece of information: purpose, usage trigger, denominator definition, and exclusion rationale. It is slightly discursive but not padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description should clarify what the agent receives in return; 'how much ... holds' is suggestive but not explicit. It also leaves the optional parameter's 'current project' default to the schema. Overall it is adequate but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents the optional 'project' parameter fully. The description adds no parameter-level detail beyond the schema, placing it at the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description answers 'how much of a project the index holds,' which identifies the tool as a coverage-measuring operation on a specific resource. It is distinct from the search/expand/context siblings in subject matter, though it does not explicitly compare itself to them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs to invoke/mention coverage before answering from the index when completeness matters, giving a concrete condition of use. It does not name the sibling alternatives or state when not to use it, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rtfm_contextA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It clearly states that only paths and line ranges are returned and that content is intentionally excluded, which is a non-obvious and important behavior. It does not mention side effects, but the read-oriented language makes mutation unlikely.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler: purpose, output shape, and the sibling alternative are each covered. Every sentence earns its place, and the most decision-relevant facts are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-style lookup tool with fully documented parameters and a clear output description, the definition is largely complete. It could be more complete by contrasting with rtfm_search, but nothing essential for selecting or invoking the tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all four parameters already have meaningful descriptions. The tool description adds no extra parameter-level meaning beyond what the schema provides, which matches the baseline for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Get sources relevant to a subject') and clarifies the result shape as paths + line ranges, not content. It distinguishes itself from rtfm_expand, but does not explicitly differentiate from sibling rtfm_search, which likely serves a similar lookup purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear alternative: 'Use rtfm_expand to read', which tells the agent to use a different tool when content is needed. It does not, however, specify when to prefer rtfm_context over rtfm_search or other siblings, so the guidance is partial.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rtfm_historyA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses the snapshot mechanism ('Each time a file is re-synced, its previous content is saved as a snapshot'), and the verbs 'Show' and 'retrieve' indicate a read-only operation. It does not discuss edge cases or restrictions, but the core behavioral trait is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two tight sentences with no filler. The primary action is front-loaded in the first sentence, and the second sentence provides essential background about snapshots. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  All parameters are documented in the schema and the snapshot mechanism is explained, but there is no output schema and the description does not state what the returned history or retrieved version looks like. An agent may not know whether it gets a list of version numbers, metadata, or file content. This is a moderate gap given the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning beyond the schema; it only mirrors the role of the version parameter. No additional detail about source, project, or version formats is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb and resource: 'Show version history of an indexed file, or retrieve a specific version.' This clearly identifies the tool's function and distinguishes it from sibling search, graph, and discovery tools by focusing on historical snapshots. The second sentence adds insight into how history is captured.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the context clear: use this tool when you need a file's version history or a particular saved version. However, it does not explicitly name alternative tools or provide when-not-to-use guidance, so it falls short of fully explicit usage routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rtfm_expandA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the primary behavior (returns file content with line numbers) and the indexed-file precondition, but it does not elaborate on other behavioral traits such as pagination behavior, the meaning of 'chunks', or any limitations. For a read-only tool this is acceptable but minimal, so a mid-range score is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The main purpose is front-loaded, and the usage guidance is immediately after. Every sentence earns its place without repeating schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter tool with no output schema, the description is short but supported by rich parameter descriptions in the schema. It clearly tells the agent when to use the tool and what the output looks like (content with line numbers). Missing details like how target or offset interact are already covered by the schema, so the description is comparatively complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all six parameters thoroughly. The description does not add extra meaning about count, offset, target, or project beyond what the schema provides, so the baseline of 3 applies. No parameter guidance is missing because the schema handles it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Read') and resource ('content of an indexed file with line numbers'), which clearly distinguishes it from rtfm_search and other sibling tools. The secondary sentence 'Use after rtfm_search' reinforces that this is the follow-up read step, and 'Like Read, but for indexed files' differentiates it from a general-purpose Read tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Use after rtfm_search', telling the agent when in the workflow to invoke this tool. The comparison 'Like Read, but for indexed files' provides an alternative and clarifies that this should be used instead of Read when the file is part of an index. This is strong, actionable usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rtfm_searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It reveals a key behavioral trait: results contain no content, only source paths and line ranges. It also indicates ranking behavior. This goes beyond the tool name and helps set expectations, though it omits details like defaults or side-effect disclaimers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences with zero filler. The action is front-loaded, the return format is stated, and the companion tool is cited. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with a fully specified input schema, the description covers the essential output behavior and points to the natural next step. It doesn't need to explain return values beyond what it states, and remaining gaps like default result count are minor and handled by the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline applies. The description adds no parameter-level meaning beyond the schema's own descriptions; all parameters are already documented adequately in the input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Search the indexed knowledge base') and explicitly specifies the output shape ('ranked source paths with line ranges, no content'). It distinguishes itself from rtfm_expand by noting that content reading is handled there, making the tool's role unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear directive: use rtfm_search to get ranked locations and rtfm_expand to read content. This is a concrete when-to-use/alternative pairing. It does not enumerate other sibling tools, but the search-versus-read distinction is sufficient for most routing decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.