Server Quality Checklist

Tool Scores

• find_coverage_gapsB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description restates 'without changing Deputy data' but adds no new behavioral traits. With annotations present, this is adequate but not enriched.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no redundant words. It is front-loaded with the core action and resources, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and 5 parameters, the description provides no details on return format, examples, or interpretation of 'shortfalls'. It relies entirely on the schema and annotations, leaving gaps for an agent to guess.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 60% (3 of 5 parameters have descriptions). The tool description adds no parameter-specific information, leaving location_ids and minimum_people unexplained. It does not compensate for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds open shifts and minimum-staffing shortfalls, specifying the verb 'find' and the resources. It does not explicitly differentiate from sibling tools, but the purpose is distinct enough to avoid confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like flag_overtime_risk or list_timesheet_exceptions. The only context is that it does not change data, which is already covered by annotations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_timesheet_exceptionsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the description doesn't need to repeat safety info. However, it adds no extra behavioral context such as pagination, number of results, or side effects beyond what annotations imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded and contains no filler. Every word earns its place, achieving maximum conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema and parameters are partially described, the description should cover return value shape or behavior. It does not mention what the output looks like, pagination, or error conditions, leaving gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 60% (3 of 5 parameters have descriptions). The description mentions 'using an explicit tolerance' which hints at tolerance_minutes but doesn't explain it or other parameters like location_ids. It adds minimal clarification beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Find' and resource 'missing or materially different timesheets', clearly distinguishing from sibling tools like find_coverage_gaps or flag_overtime_risk. It succinctly captures the tool's core purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, no prerequisites, and no context like expected use cases or limitations. The description only states what it does, not when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_availability_conflictsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent behavior. The description adds only that the tool finds overlapping rosters, which is already clear from the name. No additional behavioral traits like return format or performance implications are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, concise sentence that is front-loaded with the core purpose. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description does not explain the return value or output format. Given no output schema, this is a significant gap. The tool has 4 parameters and no output schema, so the description should clarify what the result looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75% (three of four parameters have schema descriptions). The tool description does not add meaning for parameters beyond the schema. Baseline of 3 is appropriate as schema does most of the work, but location_ids lacks schema description and tool description does not compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool finds rosters overlapping recorded unavailability or approved leave. The verb 'Find' and resources are specific. It distinguishes from siblings like find_coverage_gaps (coverage gaps) and flag_overtime_risk (overtime risks).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for detecting conflicts between rosters and unavailability/leave, but does not explicitly state when to use this tool versus alternatives. No when-not or exclusion criteria are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flag_overtime_riskA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description is consistent. It adds minimal behavioral context beyond the threshold concept, which is already hinted by the name. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded and free of unnecessary words. Every word earns its place, achieving maximum conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and a moderately complex task (5 parameters), the description omits what the output looks like (e.g., a boolean flag, a list of flagged items). It also fails to clarify the 'compare' operation details, leaving room for ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 60% schema coverage, the description does not elaborate on parameter meanings beyond the schema. For example, 'location_ids' lacks a schema description and the tool description does not compensate. The only parameter indirectly explained is 'threshold_hours' via 'configured operational threshold.'

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'compare' and a clear resource ('completed and remaining rostered hours with a configured operational threshold'). It distinguishes the tool from siblings like 'find_coverage_gaps' or 'list_timesheet_exceptions' by focusing on threshold-based risk flagging.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for flagging overtime risk but does not provide explicit guidance on when to use or avoid this tool, nor does it mention alternatives among siblings. The context is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarise_staffingA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows the tool is safe and non-destructive. The description adds specific behavioral context by stating that it summarises rostered vs completed staffing, which is beyond what annotations provide. There is no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence (9 words) that effectively communicates the tool's purpose without any wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, no output schema, and clear annotations, the description sufficiently conveys the core functionality. It is complete enough for an AI agent to understand what the tool does, though it does not elaborate on the return format or precise meaning of 'rostered and completed'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds the context of grouping by 'location and local day', which corresponds to location_ids and the date parameters, but does not detail each parameter. Since schema description coverage is 75% and the schema descriptions for required parameters are already informative, the description provides moderate additional semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'summarise' and the resource 'rostered and completed staffing' with dimensions 'by location and local day'. However, it does not explicitly differentiate from sibling tools, though the siblings have distinct purposes such as 'find_coverage_gaps' or 'flag_overtime_risk', making the purpose distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or any prerequisites. The sibling tools are listed in context but the description itself does not mention them or any conditions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.