Server Quality Checklist

Tool Scores

• find_silent_failuresC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description only lists detection criteria. It does not disclose behavioral traits like read-only nature, prerequisites, or potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence efficiently conveys purpose but lists multiple anomaly types in a somewhat dense manner. No wasted words, but readability could improve.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one optional parameter and no output schema, the description lacks details on return format, pagination, or usage context. Incomplete for a search/filter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear description for window_hours. The tool description adds no extra meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds jobs with exit code 0 flagged by silent-fail rules, using specific verb and resource. However, it does not differentiate from siblings like find_overdue_jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. The description implies usage for detecting silent failures but lacks when-not or alternative tool references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tail_job_logsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only states the result order and count, but omits read-only hint, error handling, or limitations like max lines.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words. Efficient but could include more contextual info without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple tool with 2 parameters and no output schema. Lacks details on behavior for edge cases and result format, but core purpose is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. Description hints at 'lines' parameter ('N log lines') but does not explain 'job_id' or provide format/constraints for either parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action ('Most recent N log lines'), resource ('a job'), and ordering ('newest last'). Distinguishes from siblings like get_job_status or list_jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs siblings. Does not mention exclusions or alternatives despite related tools (e.g., find_silent_failures, get_job_runs).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_job_runsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It discloses ordering, data fields, and indicators (e.g., silent-fail), but does not mention side effects, rate limits, access requirements, or return format specifics. This is adequate but not thorough for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with all key information front-loaded (recent, newest first, data fields). No wasted words; every part adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so description should explain return structure. It lists fields but does not specify if results are an array, pagination behavior (beyond limit), or error handling. Adequate for a simple list but incomplete for comprehensive understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add meaningful information beyond the schema: 'job_id' and 'limit' are already described in the schema with default and max values. No additional context for parameter usage is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the resource ('run history for a job'), action ('get'), ordering ('newest first'), and included data fields ('timing, exit code, status, silent-fail indicators, output snippet'). It effectively distinguishes from sibling tools like get_job_status or tail_job_logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like get_job_status or find_silent_failures. The description implies usage for recent runs but lacks when-not-to-use conditions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_job_statusA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; the description describes return fields but does not disclose any behavioral traits such as read-only nature, authentication needs, or cost. For a read-like tool, this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is a single concise sentence that front-loads the core purpose and lists specific details without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema), the description adequately covers return values. It could mention error handling or prerequisites, but is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage with 'job_id' documented. The description adds no further parameter information, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Detailed status for one job' and enumerates specific fields (last run, success rates, overdue state, silent-fail indicators), distinguishing it from siblings like find_overdue_jobs or get_job_runs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining a comprehensive snapshot of a single job's health, but does not explicitly state when to use this tool over alternatives or provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_overdue_jobsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core behavior (returns overdue jobs) but does not mention whether the operation is read-only, performance implications, or pagination. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is efficient and front-loaded with the core purpose, no redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description is largely complete. It could clarify what 'schedule indicates' means or the output format, but overall it provides sufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, the description adds value by explaining 'beyond a grace window' which directly connects to the grace_minutes parameter, providing context beyond the schema's technical description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns overdue jobs with a grace window, distinguishing it from siblings like find_silent_failures (different failure mode) and list_jobs (all jobs).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking missed jobs but lacks explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_jobsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite lacking annotations, the description transparently lists all returned fields, including last-run summary and overdue flags. This adequately discloses the read-only behavior and output structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose and output. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, no output schema), the description covers its functionality and output comprehensively. It could mention it as a read-only operation, but the field list suffices.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the description adds value by detailing the output fields, compensating for the absence of an output schema. It provides richer semantics than the empty input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool enumerates all known cron jobs with a last-run summary, specifying the returned fields (id, name, schedule, last run time + status, etc.). It distinguishes itself from sibling tools like find_overdue_jobs and find_silent_failures, which target specific subsets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the description implies general-purpose enumeration, it does not explicitly state when to use this tool versus siblings like get_job_status or get_job_runs. No exclusion criteria or alternative recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.