Server Quality Checklist

Tool Scores

• parts_availabilityA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It explains the effect of each parameter (e.g., 'Omit to list everything', 'return only parts at or below reorder level'), which is useful. However, it stops short of disclosing other behaviors like response ordering or pagination, though an output schema exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is minimal but complete, with a clear one-line purpose followed by an essential args list. Every sentence earns its place, and the format is front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 optional params, no nesting) and the presence of an output schema, the description provides sufficient context for typical lookups. It could be slightly richer on return behavior, but the output schema fills that gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so effectively, explaining that `query` accepts full or partial part names/SKUs and can be omitted, and that `only_low_stock` filters to parts at or below reorder level. This adds meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Look up parts stock,' which clearly identifies the tool's function with a specific verb and resource. While it doesn't explicitly distinguish from siblings like jobs_blocked_on_parts, the name and verb make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It explains the parameters but fails to mention any scenario, prerequisites, or exclusion criteria that would help an agent choose this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_jobsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral transparency. It adds useful context such as partial matching for reg and technician and the 'open' status alias, but it does not disclose traits like sorting, pagination, or read-only guarantees. The search nature implies non-destructive behavior, but it is not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single opening sentence followed by a bulleted list of parameters. Every sentence adds value, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all parameters and includes helpful examples, making it functionally complete for most use cases. It does not mention result ordering or pagination behavior, but the presence of an output schema fills in return-value details, so this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description fully compensates for the 0% schema description coverage. Each parameter is explained with valid values, examples, and semantics (e.g., status accepts specific strings and 'open' as a shortcut for unfinished jobs; reg and technician support partial matches). This provides essential meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search job cards in the workshop.' It uses a specific verb (search) and resource (job cards), and it distinguishes itself from sibling tools like vehicle_history and parts_availability, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives. It lacks any mention of sibling tools, exclusions, or prerequisites, leaving the agent without clear direction for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobs_blocked_on_partsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It clearly states the tool returns job cards blocked by short parts, but does not add further behavioral context such as ordering, filtering, or potential performance characteristics. Since it is a read-only report, the core behavior is adequately conveyed, but additional depth is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence is excellent and front-loaded. However, the second paragraph is motivational filler ('this is the question... the reason the assistant is worth having') that does not aid tool invocation. It could be removed or condensed, making the description less concise than it could be.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has zero parameters, an output schema, and a straightforward purpose, the description is largely complete. It explains what the tool does and the user scenario. The only minor gap is not specifying that it returns a list or any implicit limitations, but the output schema likely covers return details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description adds no parameter-specific semantics because none exist, but it does clarify the tool's purpose which indirectly helps the agent understand no arguments are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence, 'Open job cards that cannot proceed because a required part is short,' uses a specific verb and resource with a clear condition. This distinctly differentiates it from siblings like search_jobs and parts_availability, which have more general or alternative purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by framing it as the question a service manager asks each morning, implying this tool is for that regular check. It does not explicitly mention alternatives or exclusions, but the situational use case is clear enough to guide the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• technician_scheduleA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: day accepts ISO dates or 'today'/'tomorrow', defaults to today, technician allows partial matches, and omitting technician returns the whole workshop. This adds meaningful context beyond what annotations could provide, though it does not discuss output structure or edge cases (e.g., no bookings).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise summary followed by a bullet-like Args list. Every sentence provides useful information with no redundancy or filler. It is well-front-loaded and appropriately sized for a simple two-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (though not shown), the description need not explain return values. It covers all input options and defaults adequately. Minor omissions like timezone handling or error behavior are not critical for this simple query tool, but slightly more context about what constitutes a 'workshop booking' could improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must fully compensate. It explains both parameters in detail: 'day' format and accepted special values, and 'technician' partial matching and omission behavior. This goes well beyond the schema's bare property definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Workshop bookings for a given day,' which clearly identifies the tool's purpose as retrieving bookings for a specific day. It is specific enough to distinguish from sibling tools like search_jobs or vehicle_history, though it could be slightly more explicit by using a verb like 'Get' or 'List.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly communicates usage (to see bookings for a day, optionally filtered by technician), but it does not explicitly state when to use this tool versus alternatives, nor does it mention exclusions. Sibling tools are clearly different domains, so the context is adequate but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vehicle_historyA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description must carry behavioral info. It discloses ordering ('newest first') but doesn't explicitly state it's read-only. The read-only nature is implied by 'history,' but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with a clear one-sentence summary, a use-case example, and parameter explanation. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, has an output schema, and the description covers purpose, ordering, and parameter semantics. It's sufficiently complete for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for 'reg.' The description compensates with 'Registration plate, full or partial,' clarifying the input format and matching flexibility. This adds meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns 'full service history for one vehicle, newest first,' specifying the action and resource. This distinguishes it from sibling tools like search_jobs or parts_availability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this to answer questions like...' giving a concrete scenario (repeat visits for the same fault). It doesn't name alternatives, but the context makes the intended use clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_job_noteA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool is a mutating operation ('The only tool here that changes anything') and precisely defines the extent of mutation ('append a note and nothing else'). It does not mention any edge-case failures, but for a simple append operation this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a leading summary, a scope paragraph, and a simple args list. Every sentence serves a purpose; it is concise without being terse, and omits fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple append tool with an output schema, the description fully specifies what it does, its scope, and its parameters. The limitations are explicitly stated, so an agent has everything needed to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description compensates with an Args section. It clarifies each parameter: job_card is the ID, note is the text, author is the source. This adds meaning beyond the schema's bare titles, making parameters clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Add a note to a job card', a clear verb+resource. It further distinguishes itself from siblings by stating 'The only tool here that changes anything' and lists what it cannot do, which disambiguates it from the other read-only tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly scopes usage: 'It is narrow on purpose: it can append a note and nothing else.' It also provides when-not guidance: 'It cannot alter a job's status, its parts or its price - those are decisions that need a person.' This tells the agent when not to use it, though it doesn't name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.