Server Quality Checklist

Tool Scores

• lessons_candidateB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates the submission is for review (not direct writing), which implies a non-destructive, pending state. However, it does not disclose what happens after submission, whether the operation is reversible, or if it requires specific permissions. Adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, each adding distinct information. The first sentence explains the core action, and the second sentence mentions project scoping. No redundant or unnecessary text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 0% parameter coverage, the description is incomplete for a tool with nested objects and 4 parameters. It explains the general purpose but lacks return value expectations and deeper behavioral context. Moderately adequate for basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only mentions 'project (slug)' among the four parameters. The required 'lessons' array and optional 'task', 'agent', and 'dryRun' parameters are not explained at all. The description adds minimal value beyond the schema for parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool submits lessons for curator review rather than writing directly to the register. It distinguishes the tool's role in a curation workflow, though it doesn't explicitly reference sibling tools like lessons_apply which might directly write to the register.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use this tool (for submitting lessons for curation) and mentions setting a project slug, but it does not provide explicit guidance on when not to use it or compare it to sibling tools like lessons_query or lessons_supersede. Usage context is implied rather than directly contrasted.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lessons_queryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description alone must disclose behavior. It adds the useful detail of 'stable ids for attribution', but it does not explicitly state whether the operation is read-only, how relevance is determined, or any limits or side effects. The description is brief and leaves significant behavioral assumptions (e.g., no mutation, but not stated).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundant phrasing. It efficiently front-loads the purpose and a key output detail. While very short, it is not overly verbose, though it could arguably include more parameter context without sacrificing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With three parameters, no output schema, and no annotations, the description is insufficiently complete. It explains the basic purpose but fails to specify what the parameters do, what the response structure looks like, or any limitations. The tool's complexity is low, but the missing parameter and output details leave significant ambiguity for an agent deciding to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not mention any of the parameters (tags, limit, query) or explain their semantics. The description's 'search' implies a query parameter, but tags and limit are left unexplained, leaving the agent to guess how the parameters interact.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Search the lessons register') and the output ('return relevant lesson records with stable ids for attribution'). This distinguishes it from sibling tools like lessons_apply, lessons_candidate, and lessons_supersede, which imply different operations (applying, selecting candidates, superseding).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool is for searching lesson records, but it does not explicitly state when to use it instead of the sibling tools, nor does it mention any exclusions or prerequisites. The usage context is clear from the verb 'Search' but lacks explicit alternatives or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lessons_applyA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It discloses the tool is telemetry-only (non-functional recording), but does not clarify the behavior of the dryRun parameter, error handling, or whether the operation is idempotent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and contains no redundant words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, no output schema, and no annotations, the description is adequate for a simple telemetry tool but lacks explanation of the dryRun parameter, return value, and the relationship between lessons and tasks, leaving some context incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for lessonIds, task, and outcome (including enum semantics), but does not explain agent, dryRun, rationale, or outcomeNote, leaving gaps for 4 of 7 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool records which lesson IDs influenced a task and specifies it's for telemetry only. It distinguishes from siblings like lessons_query (retrieval) and lessons_candidate (suggestion) by emphasizing recording, but does not explicitly differentiate from lessons_supersede.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for recording influences and advises when to include the outcome field, but provides no explicit guidance on when not to use this tool or when alternatives like lessons_query or lessons_supersede are preferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lessons_supersedeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It states that the tool never deletes (non-destructive), updates statuses (superseded/deprecated), and affects query results. It also implies a role restriction. However, it does not disclose potential side effects, idempotency, or error handling, leaving some behavioral traits undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the key action and role restriction. Every sentence adds value: role, action, behavioral guarantee, and effect on queries. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and no annotations, the description covers purpose, role, and core behavior but omits details on dryRun, reason usage, and return value. It does not explain the difference between superseded and deprecated statuses beyond the pointer. While adequate for a curation tool, it leaves gaps for an agent to fully understand the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning for the status and supersededBy parameters by explaining they mark lessons as superseded (with a pointer) or deprecated. It does not clarify the dryRun parameter, the reason parameter (beyond being required), or the ids parameter. Since schema description coverage is 0%, the description partially compensates but lacks full parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'retire lessons by judgment' with specific verbs (retire, marks) and resource (lessons). It distinguishes from sibling tools like lessons_query by explicitly noting that retired lessons stop appearing in lessons_query, and from lessons_apply/lessons_candidate by being a retirement action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies 'Curator only' as a role restriction, and contrasts with deletion by stating 'Never deletes'. It gives context for when to use this tool (to retire lessons) and the effect on visibility. However, it does not explicitly mention alternatives or when not to use it, though the sibling tool names hint at other actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.