Server Quality Checklist

Tool Scores

• list_queuesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so description's mention of 'the inspector can see' adds marginal value. Mention of backend tagging provides some output behavior detail but lacks disclosure of pagination, rate limits, or access restrictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately conveys the tool's action and key feature (backend tags). No extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description partially compensates by noting output includes backend tags, but doesn't specify other returned fields or structure. Adequate for a simple list tool but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage for the single parameter 'backend' (with enum). Description adds no extra parameter semantics beyond schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every queue visible to the inspector, with a specific detail about tagging with backend type. This distinguishes from sibling tools focused on jobs or queue statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, such as queue_stats or other queue-related tools. The description only states functionality, not context of use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queue_statsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint: true, so the agent knows this is a safe read operation. The description adds behavioral context by stating 'using the backend's own state names', which warns that state names are backend-specific (asynq vs bullmq). However, it does not disclose error behavior when the queue is missing or missing backend parameter, but given the simplicity of the tool, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 13 words that is front-loaded with the verb and resource. Every word is necessary and contributes to understanding. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema, the description could have briefly mentioned that the output is a mapping from state names to counts. However, the tool is simple (2 params, 1 required), and the core purpose is clear. The absence of output details is a minor gap but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not elaborate on individual parameters beyond the schema. It implicitly references the 'queue' parameter but does not add meaning or context for the 'backend' parameter (e.g., when it is optional).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Report' and clearly identifies the resource: 'number of jobs in each state for a queue'. It also notes 'using the backend's own state names', which adds precision and distinguishes the tool from siblings like list_jobs or get_job that deal with individual jobs rather than aggregate state counts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining queue statistics but provides no explicit guidance on when to use this tool versus alternatives such as list_jobs or list_queues. There is no mention of prerequisites, when not to use it, or how it differs from sibling tools like get_job or retry_job.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jobA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, and the description aligns with a read operation. The description adds behavioral context beyond annotations by noting that binary payloads are base64-encoded and flagged. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that are concise and front-loaded. The first sentence communicates the core purpose and visible outputs, the second adds an important detail about binary encoding. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately explains what the tool returns with specific fields and handles binary payloads. Given no output schema, this is sufficient for a single-job detail retrieval. However, it could mention the return format (e.g., JSON object) for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for each parameter (id, queue, backend). The description does not add additional meaning to parameters beyond what the schema already provides, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Fetch' and resource 'full detail for one job', listing specific fields (payload, attempts, retry ceiling, last error, timestamps). This clearly distinguishes it from siblings like list_jobs (list of jobs) or delete_job (delete).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving detailed information about a specific job but does not explicitly state when to use this tool versus alternatives like list_jobs, retry_job, or delete_job. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_jobA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds context beyond the destructiveHint annotation, stating the deletion is permanent and cannot delete active jobs. It also mentions replicating backend behavior, which is useful for understanding side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences, front-loading the main action and constraints. No superfluous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a delete operation, the description covers key constraints and backend fidelity. It lacks information about the return value or error behavior, but the context is sufficient for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already describes all parameters. The description does not add additional semantics beyond what is in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'permanently delete' and the resource 'job from a queue'. It distinguishes from siblings like get_job, list_jobs, retry_job by specifying deletion and noting that active jobs cannot be deleted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Active jobs cannot be deleted', providing a clear when-not condition. However, it does not explicitly guide when to use this tool over alternatives like retry_job or list_jobs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retry_jobA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotentHint=true and destructiveHint=false, so the description adds value by explaining the effect (moving to pending queue) and faithfulness to backend retry. It does not contradict annotations. Would benefit from stating constraints like job must be in a retryable state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose clearly, second reinforces behavioral accuracy with backend reference. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema but annotations cover safety. The description explains the outcome (job moved to pending queue). For a simple mutation, this is mostly complete, though returning a success indicator would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for each parameter (id, queue, backend). The description does not add new meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Move a failed or dead job back to the pending/wait queue') and target resource ('failed or dead job'). It distinguishes itself from sibling tools like delete_job (removal) and get_job (reading) by focusing on retrying a job.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrying failed/dead jobs and references backend retry functions, providing context. However, it lacks explicit guidance on when not to use the tool (e.g., if the job is already pending) or direct comparison with alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_jobsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true; the description adds that results are paged and include id, type, attempts, and a truncated last error. This provides behavioral context beyond the annotation, such as truncation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the core purpose, the second adds backend-specific state details. It is front-loaded and contains no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters and no output schema, the description covers the return fields and pagination hint. It is sufficient for a list tool, though it could mention default sorting or ordering, but not necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by enumerating backend-specific states and clarifying the optionality of the backend parameter. This supplements the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists jobs filtered by state, with pagination. It specifies the verb (list), resource (jobs), and scope (by state, paged). It uniquely distinguishes from sibling tools like delete_job or get_job.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit valid states for each backend, guiding proper use. It does not include explicit when-not-to-use or alternatives, but the context is clear enough for correct invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.