Server Quality Checklist

Tool Scores

• codemap_statusA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral disclosure burden, and it does partially by clarifying that the tool reports existence, mtime, and completeness rather than returning CodeMap content. It does not state whether the operation is read-only, what happens when the CodeMap is missing, or any authentication or error behavior. This is adequate for a likely innocuous status check but leaves gaps around edge-case behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that packs the resource, the condition (committed), the parameters (operator + architecture), and the returned attributes (existence, mtime, completeness) with no wasted words. Everything present is informative and directly relevant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter schema and no output schema, the description provides the core return semantics, but it does not describe the response shape, success/failure indicators, or behavior when no committed CodeMap exists. It is functionally complete for a basic status check but could benefit from a note on the return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so by mapping the two parameters to 'this operator + architecture', indicating that 'project' likely refers to an operator and 'architecture' to an architecture. It does not define valid values, formats, or the relationship between the fields, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear, specific purpose: it checks whether a committed .uo CodeMap exists for a given operator and architecture, and reports mtime and completeness. It is not a tautology and goes beyond the name by naming the exact resource and attributes. However, it does not explicitly contrast itself with siblings like query_codemap or codemap_doctor, so differentiation is implied rather than stated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The wording 'Whether a committed .uo CodeMap exists' implies an agent should use this tool to check existence/status, but there is no explicit guidance on when to reach for it instead of query_codemap or codemap_doctor. It lacks any mention of preconditions, alternatives, or exclusions, so the agent must infer usage from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• codemap_doctorA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the phrase 'Check whether' credibly signals a non-mutating diagnostic and the colon list reveals exactly what environment conditions are inspected. It does not spell out the result format or whether the check can have side effects, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence states the purpose before a colon-delimited list of dependencies. There is no filler or duplication of the schema; every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity diagnostic with 100% schema coverage, the description gives an agent enough to select and call it correctly. The main gap is the absence of an output schema or a stated return shape (e.g., boolean vs. detailed report), but the core invocation context is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameters are already documented. The description adds context by confirming that 'project' is the operator directory and architecture is one of the checked conditions, but it does not add new syntax or semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('Check whether this machine can build an AscendC operator CodeMap') and a concrete resource, with an explicit list of checked prerequisites (CANN headers, libclang, operator directory, architecture). This clearly distinguishes a preflight diagnostic from the sibling index/update/query tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The use case is implied: call this when you need to know whether the machine is ready to build a CodeMap before running a build or index operation. However, it does not explicitly name alternatives or say when not to use it, leaving some routing to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_codemapA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure and does state that the tool is read-only and accepts only four shapes. It does not cover invalid-input outcomes or authentication expectations, but the main safety-relevant behavior is clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the read-only nature and allowed shapes. Every sentence adds a useful constraint or clarification, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main query patterns but leaves line_end undefined, does not explain project/architecture usage, and the schema's 'Required.' note on architecture conflicts with the declared zero required parameters. An agent can use the main forms but may be uncertain about combinations and output behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the schema by explaining pattern syntax, the file+line combination, and the omission rule for the index case. It does not describe line_end or clarify the requiredness of architecture and project, but the core query semantics are well covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific read-only query operation on the operator CodeMap and enumerates the four accepted input shapes. The read-only qualifier sharply differentiates it from mutation siblings like update_operator.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit valid invocation forms and explicitly says not to pass natural-language sentences. It does not name alternative tools for other use cases, but the context is clear enough for an agent to know how to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_operatorA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden. It explains the pipeline stages and the rebuild behavior, implying a mutating operation. It also warns against calling on MCP connect. However, it doesn't explicitly state side effects like overwriting an existing .uo or the nature of the output, which would add more transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences deliver the purpose, pipeline, preconditions, and alternative routing without fluff. Critical usage constraints are front-loaded and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only two required params and no output schema, the description covers the main decision points well: what the tool does, when to call it, and when not to. Minor gaps remain around post-conditions (e.g., what a successful run produces) and whether 'rebuild' is destructive, but these are not critical for a tool with such strong usage guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters are already documented with types and descriptions. The description only restates that project and architecture are required, adding no additional semantic detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Build or rebuild the operator CodeMap' with a pipeline (prepare → extract → analyze → commit). It distinguishes itself from sibling update_operator by explicitly stating when to use that alternative instead.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit conditions for use: 'only when the user asks to index or no .uo exists' and 'Do not call on MCP connect.' It also names the alternative (update_operator) for the opposite case, leaving no ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_operatorA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that this is an incremental rebuild of changed layers, that an existing .uo is required, and that it must not be called on MCP connect. It does not detail side effects or reversibility, but provides substantial behavioral context beyond a bare 'update' label.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise, purposeful sentences. The core purpose is front-loaded, followed by two essential constraints and a clear pointer to the alternative tool. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no output schema, the description covers prerequisites and usage boundaries well. However, it does not describe the return value or outcome of the plan/confirmation flow, and it doesn't state what happens if confirm_scope is false. This leaves a small but meaningful gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all three parameters (project, architecture, confirm_scope) are already documented in the schema. The description adds no extra parameter-level meaning, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'incrementally refresh an existing CodeMap', and explains the internal pipeline (detect → plan → rebuild changed layers). It clearly distinguishes itself from index_operator ('only when no .uo exists'), leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use the tool ('Requires an existing .uo'), when not to use it ('Do not call on MCP connect'), and names the alternative: 'Use index_operator only when no .uo exists.' This is direct and actionable guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.