Server Quality Checklist

Tool Scores

• reassign_deliveriesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does reveal that the operation mutates assignments and that unassigned deliveries are listed, but it does not explain whether existing route assignments are overwritten, whether the change persists immediately, or what side effects occur on current plans. This is a meaningful transparency gap for a mutating tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single tight sentence with the action verb front-loaded and the output behavior separated by a semicolon. There is no filler or repetition; every word adds information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema and no annotations, so the description must explain both invocation context and return behavior. It only partially explains results by saying what did not fit, but it does not describe the successful reassignment output, nor does it clarify how this relates to the sibling route-planning and insertion tools. The agent is left uncertain about side effects and next steps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It maps delivery_ids to 'the given deliveries' and date to 'for a date,' and adds context about 'active vehicles.' However, it does not explain the expected date format, what kind of delivery IDs are expected, or how active vehicles are chosen.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific mutating verb ('Redistribute') and names the resource ('the given deliveries') and scope ('among active vehicles for a date'). It also hints at a distinctive partial-failure output ('lists what did not fit'), which separates it from plain listing tools. It does not explicitly differentiate itself from siblings like plan_routes or commit_insertion, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to choose this tool over alternatives such as plan_routes, evaluate_insertion, or commit_insertion. It neither states prerequisites (e.g., needing active vehicles or pre-existing deliveries) nor says when not to use it. Any usage context is only implied by the word 'redistribute.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• commit_insertionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of behavioral disclosure. It does reveal that downstream ETAs are rewritten, but it omits persistence/irreversibility, failure behavior, permission requirements, and whether the route is modified in place.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It front-loads the action and immediately states the key consequence, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It does not describe return behavior, error cases, side effects beyond ETA rewriting, or how this tool relates to evaluate_insertion, so an agent may not call it correctly in all contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, and the description does not add meaning beyond the parameter names. It does not clarify delivery_id or route_id semantics, nor does it explain position bounds or what happens on an invalid position.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Insert'), names the resource (a delivery into a route), and specifies the action's effect (rewrites downstream ETAs). This clearly differentiates it from evaluate_insertion, which would assess but not rewrite ETAs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. Given the sibling evaluate_insertion, an agent must infer when to commit versus evaluate, and there are no exclusions, prerequisites, or validation conditions mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_route_detailB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It makes clear this is a read-style operation returning route details, but does not mention what happens for invalid or missing route_ids, permission requirements, or whether cumulative load is computed or stored. The verb 'get' implies read-only, which is reasonable but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It front-loads the core purpose and immediately lists the key output dimensions, making it easy to scan while conveying the essential value of the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read tool, the description gives enough output detail to understand what the tool returns. However, there is no output schema and no mention of where route_id comes from or when this tool is appropriate, leaving some practical gaps for an agent deciding to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate for the undocumented route_id parameter. It does not mention route_id at all, its origin, or how it relates to plan_routes. The parameter name is self-explanatory, but the description adds no semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('get') and resource ('route detail') and lists concrete output contents: stop-by-stop sequence, ETA, and cumulative load. It is clear enough to distinguish from list_deliveries and list_vehicles, though it does not explicitly differentiate from export_route_sheet.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus siblings such as export_route_sheet or evaluate_insertion. The phrase 'planned route' vaguely implies it should be used after route planning, but no explicit condition or alternative is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mark_vehicle_out_of_serviceB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It reveals the noteworthy side effect of returning orphaned deliveries and clarifies that these were previously assigned, but it does not disclose whether the operation is reversible, what exactly becomes inactive, or what happens to the deliveries after return.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one efficient sentence with the primary action front-loaded and a useful parenthetical clarification. Every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description omits essential selection context, parameter explanation, and side-effect detail. It is better than a tautology but still incomplete for a mutation tool with two parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description adds no meaning for either parameter. 'vehicle_id' is implied by 'vehicle', but 'reason' is completely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action and resource ('Mark a vehicle inactive') and adds a distinguishing side effect ('return its orphaned (previously assigned) deliveries'), which clearly separates this operation from sibling list/plan/update tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool instead of alternatives such as reassign_deliveries or update_delivery_status. The intended context is only implied by the action name, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_route_sheetB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states the action and output formats, omitting any details about side effects, required permissions, error handling, or whether the operation is read-only. For an export tool, this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that efficiently conveys the core purpose. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description covers the basic function but lacks details on return value, error conditions, or prerequisites. It is minimally adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions (0% coverage), so the description should clarify parameter meaning. It only mentions 'for a route' (route_id) and 'as markdown or CSV text' (format), which adds little beyond the schema's structure. The required nature of route_id and default of format are not mentioned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: rendering a printable route sheet for a specific route, with explicit output formats (markdown or CSV). It distinguishes itself from sibling tools like get_route_detail by focusing on printable output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool compared to siblings, nor does it mention any prerequisites or scenarios. An agent must infer its usage from the name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_delivery_statusB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. It indicates the operation is a mutation ('Set') but does not disclose side effects, validation rules, failure behavior, or whether status transitions are restricted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It states the action, the resource, and the value domain efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool, the description is minimally adequate. However, without annotations, output schema, or usage context, it leaves gaps around status transition rules and expected behavior in edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameter meaning. It only restates the enum values already present in the input schema and provides no additional meaning for delivery_id, its format, or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Set') with a specific resource ('delivery's status') and enumerates the four valid status values. This makes the tool's purpose unmistakable and clearly distinguishes it from sibling tools like reassign_deliveries or mark_vehicle_out_of_service.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus siblings, when status transitions are allowed, or what the expected workflow is. The only implied usage is the generic action stated in the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_deliveriesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the safety burden itself. 'List' implies a read-only operation and the fields describe what is returned, but the description does not explicitly confirm it has no side effects, nor does it mention optional filtering behavior or output characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence contains the core action, scope, and return fields with no filler or redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter list operation, this is minimally sufficient: an agent knows the date is required and what fields are returned. However, it leaves the optional status filter unexplained and provides no reassurance about read-only behavior, pagination, or empty results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only 50% of parameters carry schema descriptions, and the description does not compensate: the required 'date' is repeated but the optional 'status' parameter is not described beyond its enum values. No explanation of status as a filter is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('List'), a resource ('deliveries'), a scoping condition ('for a date'), and the returned fields (weight, address, time window). This clearly distinguishes it from siblings like list_vehicles or plan_routes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The date condition implies this is for fetching a delivery manifest for a specific day, but there is no explicit statement of when to prefer it over siblings such as get_route_detail or export_route_sheet. No alternatives or exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_vehiclesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the burden of behavioral disclosure. The verb 'List' and the listed output fields signal a read-only retrieval operation, but the description does not disclose filtering behavior, whether inactive vehicles are included by default, or pagination/ordering behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence that front-loads the action and resource, then lists the key output attributes. There is no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two optional parameters and no output schema, the description covers the main output fields but omits parameter behavior and any filtering context. It is adequate for a basic call but not fully complete for an agent deciding how to invoke it with parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain depot_id or only_active. The parameter names are somewhat self-explanatory, but the agent receives no guidance about filtering semantics, the default of only_active, or how depot_id affects results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('List fleet vehicles') and clarifies the output focus: capacity (kg/units), depot, and driver shift. This clearly distinguishes it from sibling tools like list_deliveries or plan_routes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied by the verb and resource, but the description does not explicitly state when to choose this tool over alternatives or mention any exclusions. Sibling tools such as list_deliveries are not referenced for comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plan_routesA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return contents (per-vehicle stop sequence, ETA, distance, duration, unassigned deliveries with reasons) and the important side effect that the plan is persisted. It does not mention overwrite behavior or computational cost, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and well-structured: one sentence for purpose, one for outputs, one for side effect. Every sentence adds useful information with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's core purpose, outputs, and persistence side effect. However, it omits parameter semantics and when-to-use guidance relative to sibling planning tools, leaving an agent to infer important details about optional inputs and behavior for a 4-parameter tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 50%, and the description does not compensate for the undocumented parameters. It mentions 'date' inline but does not explain objective, vehicle_ids, or traffic_factor beyond what the schema already provides. The enum values and the filtering semantics of vehicle_ids remain unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Plan routes for the fleet on a date' and identifies the problem type (CVRPTW). It clearly distinguishes itself from siblings like get_route_detail, evaluate_insertion, and export_route_sheet by focusing on plan creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when fleet routes need to be computed for a date, and the note about persisting the plan hints at a side effect, but it does not explicitly state when to use this tool versus alternatives like evaluate_insertion or commit_insertion. No exclusions or alternative routing are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluate_insertionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the computed outputs: marginal cost, best position, and violation checks. However, it does not explicitly state that this is a non-mutating evaluation, nor describe the response shape beyond the listed outputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one dense, well-structured sentence. It front-loads the core outputs, then states the input requirements. No filler or redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description covers the main inputs, the decision context, and the key returned information. It could add date format guidance or clarification of 'active routes', but the provided description is largely sufficient for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds important meaning by explaining that date is the context, and that either delivery_id (existing) or a new_delivery object must be provided. It does not detail nested new_delivery fields, but the schema field names are already self-explanatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific purpose: computing marginal cost, best insertion position, and capacity/time-window feasibility for a delivery across active routes. The verb is implied by the tool name but the description itself is action-oriented and clearly differentiates from commit_insertion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly communicates when to use the tool: when assessing insertion cost and feasibility for a given date. It explains the two input modes (existing delivery_id or new_delivery object), but it does not explicitly say 'use this before commit_insertion' or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.