Server Quality Checklist

Tool Scores

• atlas_graphA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses that the tool reads the derived graph.json produced by atlas_index and does not rebuild the graph, indicating a read-only dependency on a precomputed artifact. This is useful context, though it omits details like staleness or side effects (which are minimal for a read-only query).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the main verb and resource front-loaded. The second sentence adds essential context about when to use and the dependency on atlas_index. Every phrase earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of a full output schema and complete parameter descriptions, the description does not need to explain return values or parameter details. It covers the core purpose, usage context, and dependency, which is sufficient for a tool of this complexity. Minor gaps like explicit exclusions or performance notes are not critical because the schema fills in details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage, with descriptions for every parameter including conditional requirements (e.g., 'Required for path'). The description adds no additional parameter-level information, so it sustains the baseline score of 3 but does not elevate it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool queries the derived knowledge graph for hubs, paths, or neighborhood explanations. The verb 'query' and the specific resource ('derived knowledge graph') distinguish it from siblings that operate on raw indices or briefs. It also specifies the graph is derived from atlas_index, providing unique identity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to call this tool when the question concerns connectivity, rationale, or centrality, which gives clear usage context. It also notes it does not rebuild the graph, implying it is for read-only queries. However, it does not explicitly name alternative tools or contrast with them, though the 'call directly' phrase hints at direct selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_briefA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so excellently. It discloses deterministic derivation, response size capped independent of repo size, ranked/capped lists, confidence levels with declared vs inferred, known warning types, and staleness detection limitations including the inability to detect uncommitted edits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but every sentence earns its place. It is front-loaded with a one-sentence summary, followed by explicit usage instructions, output details, and limitations, without redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the absence of annotations, the description covers all necessary context: what it does, when to call it, what it returns, how to interpret results, known edge cases, and relationships to sibling tools. The output schema existence further reduces the need to explain return types, but the description goes beyond that anyway.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add new meaning about the `level` parameter beyond what the schema already says, though it enriches the overall understanding of the response shape (ranked, capped lists) which indirectly informs parameter expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Get a pre-computed, token-bounded briefing that orients you in an unfamiliar project.' It clearly distinguishes the tool from siblings by explicitly naming atlas_search, atlas_graph, and atlas_index and stating what each is for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance ('Call this FIRST, once, when you start working on a project you do not already know'), what not to use it for, and names alternative tools for specific tasks. It also warns against repeated calls and notes when the briefing changes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_indexA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description fully discloses incremental behavior, skip of unchanged files, async/sync modes, and dry_run behavior. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured, concise with clear sections. Each sentence adds value; no fluff. Front-loaded with purpose, then usage notes, then behavioral details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given four parameters, async behavior, and an output schema, the description covers all necessary context: workflow, behaviors, edge cases (dry_run, full rebuild). Output schema exists, so return values need not be described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers all parameters (100% coverage). Description adds value by explaining how parameters affect execution (e.g., dry_run returns candidates, non-empty paths with full=false is synchronous) and usage patterns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool indexes source code and documents into a local search index, using specific verbs and resources. It distinguishes from the sibling tool 'atlas_search' which presumably searches, not indexes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance on when to use (first time, after staleness) and when not (use dry_run first, ask user). Provides workflow instructions and distinguishes between async and sync execution based on parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the transparency burden. It discloses the hybrid search mechanism (vector + BM25, RRF fusion), default metadata-only return, two-pass pattern, token-efficiency consideration, include_content behavior, and the actionable error when the index is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately sized but every sentence adds unique value, from purpose to usage to behavioral details to error handling. It is well-structured with a clear progression and imperative guidance, not wasting words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex search tool with 7 parameters and an output schema, the description covers all essential aspects: what it searches, how it works, what it returns by default, how to use it in a token-efficient pattern, and what to do if the index is missing. It also integrates sibling tool context, making it self-sufficient for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics beyond the schema by explaining query input as 'natural language or exact symbols' and elaborating the default return fields and the purpose of include_content, going beyond the schema's terse descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches code and documents in the project's local index, establishing it as the primary search/find tool. It distinguishes itself from siblings by explicit 'FIRST tool' positioning and naming atlas_brief as the alternative for getting bearings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: use before broad file reads or grep, use atlas_brief for unfamiliar projects, and do NOT call atlas_status first. It also explains the error path related to atlas_index, giving clear when-to-use and when-not-to-use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_statusA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It clearly states the tool never indexes anything, and it discloses important behavioral details such as the reindex lock ('reindexing' true when another process holds the lock), staleness semantics, and index resolution modes. This gives the agent a strong understanding of side effects and edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear one-sentence purpose, followed by usage guidance and a detailed return list. Every sentence serves a purpose: usage restrictions, non-side-effect guarantee, and return field explanations. It is appropriately sized for the tool's diagnostic nature.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully compensates for the lack of annotations by documenting the output fields, including edge-case values like index_resolution and is_stale. It is complete for a diagnostic tool with no parameters and no side effects, and it provides enough context for the agent to interpret results correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already covers everything. The description adds no parameter-specific details, but with no parameters the baseline is 4, and the description's return-value documentation compensates for any lack of parameter context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Get diagnostic metadata and health status of the local vector index') tied to a clear resource (local vector index). It distinguishes itself from sibling tools by explicitly stating it is not a precondition for atlas_search/atlas_brief and never indexes anything.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use (explicit diagnostics or deciding whether atlas_index should run) and when not to use (not a precondition for search/brief; call those directly). It also names an alternative tool (atlas_index) and clarifies that search/brief raise errors themselves if the index is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.