Server Quality Checklist

Tool Scores

• menuA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true and openWorldHint=false, and the description does not contradict these. It adds useful behavior: without keeping catalog resident, with no query lists the catalog, and includes a state_changing field in returned rows. This goes beyond the annotation by describing the read-only nature and output semantics. Solid but not exhaustive (no mention of auth or rate limits), hence a 4.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences front-load the purpose, then immediately give output format and a pairing tip. Zero fluff; every sentence earns its place. The parenthetical 'menu(query?)' is compact and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given it's a read-only tool with no output schema, the description covers what it returns (compact rows with specific fields), how to invoke it with or without query, and how to follow up with 'order'. It doesn't mention the limit parameter but that is trivially covered by the schema. It could mention error conditions or an example, but for a simple lookup tool this is quite complete. A 4 is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for both parameters, so the baseline is 3. The description adds value by showing the query param in shorthand 'menu(query?)' and explicitly stating that with no query it lists the catalog, which clarifies default behavior beyond the schema's 'Optional' note. This is a meaningful addition, pushing it to a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Discover' and the resource 'catalog actions', and specifies exactly what it returns (compact rows with fields). It also explains the no-query behavior. It doesn't explicitly differentiate from all five siblings, but the pairing with 'order' gives context. Slightly below a 5 because it doesn't enumerate alternatives beyond 'order'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to pair with 'order' to dispatch an action, which implies when to use menu (discovery) vs order (execution). It also explains the no-query listing behavior. However, it doesn't mention when not to use it relative to other siblings like 'route' or 'set_tool_tier'. Clear enough guidance for the most relevant alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• routeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations provide readOnlyHint=false and openWorldHint=true, so the tool is not read-only. The description builds on this by disclosing that execute=true dispatches the top recommendation and returns the result, collapsing the discover-then-call flow into one round-trip. It also clarifies that without execute it returns recommendations only, giving the agent a clear behavioral model without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, then explains the execution option and a usage hint for context-gathering intents. Every sentence earns its place; no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a router that can optionally execute, the description covers the main behavior (recommendations vs. execution), the return type (ranked recommendations with argument templates), and even suggests related tools for specific intents. It does not detail error handling or authentication, but these are not essential given the tool's simplicity and the presence of a schema with parameter semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema contains full descriptions for all 4 parameters (100% coverage), including task, repo, model, and execute. The description does not add new parameter-specific semantics; it reiterates the execute behavior already documented. Given the schema covers meaning, the description adds no v、alue here, aligning with the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Map a natural-language task to the best catalog action(s)' – a specific verb + resource that clearly distinguishes it from sibling tools like set_tool_tier or menu, which are direct actions. It also describes its primary output (ranked recommendations) and the optional execution path, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clarifies when to use the tool by indicating it is for natural-language task routing and explicitly recommends assemble_task_context / plan_turn for context-gathering intents. However, it does not explicitly state exclusions (e.g., 'use when you know the exact action'), though this is strongly implied by the router nature. It provides actionable context without being prescriptive about alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jcodemunch_guideA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description consistently describes a read operation with no contradiction. It adds valuable context beyond the annotations: idempotency, that no repo context is required, and that output dynamically matches the current tool surface, tier, and disabled_tools configuration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and each subsequent sentence earns its place: the drift-prevention use case, idempotency/no-repo-context, and the config-matching behavior. It is slightly dense — four ideas in one block — but efficient with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only tool with no output schema, the description is thorough: it covers return content, the rationale, safety (idempotent, read-only), and the config-sensitive output behavior. The only minor omission is a concrete sketch of the returned snippet's shape, but given tool simplicity this is not a material gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is an empty object (100% coverage trivially), so the schema leaves nothing to explain. Per rubric, a zero-parameter tool gets a baseline of 4, and the description adds the useful note that no repo context is required, reinforcing the no-input contract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Return') plus a precise resource: the version-current CLAUDE.md/AGENT.md policy snippet generated by `jcodemunch-mcp claude-md --generate`. This is clearly distinct from its siblings (set_tool_tier, announce_model, menu, order, route), which are all operational actions, so an agent can disambiguate without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains why and when to call it: to keep a one-line CLAUDE.md instead of a static snippet that drifts from the installed version, and notes 'no repo context required' so it can be invoked standalone. It does not explicitly name alternatives or state when not to use it, but the clean use-case motivation covers the main guidance need.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• orderA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite openWorldHint=true and readOnlyHint=false in annotations, the description adds valuable behavioral detail: it is read-only by default, requires allow_state_change=true for state-changing actions, and refuses execution/file-write verbs. This goes beyond the annotation flags and helps an agent predict side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but efficient. Each sentence adds functional value: the call pattern, read-only default, state-change opt-in, exploration recommendation, and menu/route routing. It could be slightly trimmed, but the structure is logical and front-loaded with the primary usage pattern.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generic dispatcher with no output schema and dynamic behavior, the description is impressively complete. It covers invocation, safety constraints, suggested use cases, and fallback tools for discovery. It does not describe return values, but given the open-world nature of dispatched actions, that is an acceptable omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics by showing the call pattern order(action, args), explaining that args are passed 'exactly as you'd pass them directly,' and clarifying allow_state_change with a concrete example (index_repo). The compress=true hint further enriches parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb-resource pair ('dispatch any jcodemunch action by name') and positions itself as 'the single-verb front door to the full tool catalog.' It distinguishes itself somewhat from siblings by noting that 'menu' discovers actions and 'route' picks one, though it could more sharply contrast with 'route' as a dispatcher vs. a router.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Excellent guidance: it explicitly recommends order('get_ranked_context', ...) for exploration questions over chained search/outline/source hops, suggests adding compress=true, and directs users to 'menu' for discovery or 'route' for task-based selection. It also states that execution/file-write verbs are refused, giving a clear boundary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• announce_modelA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses multiple behavioral traits beyond the annotations: idempotency ('a second call with the same model is a cheap no-op'), the fuzzy tier resolution order, and the side effect of narrowing the tool list. These details help the agent anticipate what will happen when it calls the tool, well beyond the sparse annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and purpose, then detailed resolution logic, and ends with usage guidance. Every clause carries value, and there is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description covers purpose, effect, idempotency, and when to use it. Nothing an agent needs to decide whether to call it or to understand the outcome is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides a clear description of the model parameter with an example ('claude-haiku-4-5'), so schema coverage is 100%. The description adds little beyond restating that the agent reports its model identifier, which is already implied by the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('self-reports') and resource ('active model identifier'), and clearly explains the consequence: the server resolves to a tier and narrows the exposed tool list. It also distinguishes itself from plan_turn by name, so an agent can tell them apart without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given: 'Prefer calling plan_turn(model=...) for routine per-task use; use announce_model as a fallback when plan_turn is not appropriate.' This tells the agent exactly when to use this tool and when to use the alternative, leaving nothing to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_tool_tierA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=false and openWorldHint=false, so agents know this is a mutating, closed-world operation. The description adds that it narrows/widens the exposed tool list and applies to the current session, which is useful context beyond the annotations. It doesn't disclose every side effect (e.g., persistence), but it's adequate given the annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, then usage guidance. Every word earns its place with no filler or redundancy. Efficiently structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter, no-output-schema tool, the description fully covers what it does, when to use it, and the allowed parameter values. There is nothing critical missing for a correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has zero description coverage for the parameter, but the description lists the enum values ('core', 'standard', 'full') and explains their effect (narrow/widen tool list). This fully compensates for the schema gap, making the parameter's meaning clear without needing to open the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states a specific action (override tier for current session) and the resource (exposed tool list), and explicitly names the three allowed values. It also distinguishes itself from plan_turn, the routine alternative, so an agent can immediately tell what this tool is for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit routing: prefer plan_turn(model=...) for routine per-task use, use set_tool_tier only for explicit overrides, and provides a concrete example (escalate mid-task to 'full' after a capability-gated failure). This leaves no ambiguity about when to pick this tool over the alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.