Server Quality Checklist

Tool Scores

• jobboss2_create_reportB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal a non-read-only, open-world, non-idempotent operation. The description adds valuable transparency by stating that ECI's OpenAPI document does not define the request body fields or response schema, which is critical for an agent deciding whether it can trust schema validation or needs other knowledge. It does not, however, disclose whether report generation is asynchronous or what side effects occur beyond submitting the request.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The action is stated first, followed by a necessary caveat about the missing schema definitions. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The schema caveat is important, but for a submit operation with an unconstrained object parameter, the agent still lacks guidance on what fields to include, what this request means for report generation, or how to follow up (e.g., using jobboss2_get_report). The output schema and annotations help, but the description alone is not fully sufficient for confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single generic `data` object with 0% description coverage. The description only notes that fields are undefined by ECI's OpenAPI document, which explains why the schema is vague but gives the agent no concrete meaning, examples, or guidance for constructing the `data` payload.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Submit a JobBOSS2 report request.' It clearly identifies this as a report-submission action rather than a retrieval or generic list operation. However, it does not explicitly distinguish itself from siblings like jobboss2_create_report vs jobboss2_get_report or jobboss2_create, leaving some ambiguity about whether it triggers generation or just records a request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no when-to-use guidance and no contrast with alternatives such as jobboss2_get_report for retrieving results or jobboss2_create for general creation. The use case is only implied by the title and name, not explained in the description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_listA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds meaningful behavioral context beyond the annotations by documenting the filter grammar and operator set, and by noting offset paging. Since readOnlyHint and idempotentHint already cover the safety profile, the description does not need to restate those traits. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with the primary operation stated first and the essential filter grammar in a follow-up sentence. No sentence is wasted, and the structure makes the most material information easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description names the main capabilities and the output schema covers return shape, but the tool is a generic, highly parameterized list endpoint with a large resource enum. It does not direct the agent to jobboss2_describe_resource for resource-specific field definitions, nor does it clarify when list should be preferred over get. This is adequate for basic invocation but not fully robust for correct tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description does add value by explaining the field[operator] filter syntax and the available operators, and by tying skip/take to offset paging. However, it does not fully define parameter behavior: sort prefix conventions, field sourcing, take limits, and filter value semantics are left mostly to schema inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and a JobBOSS2 resource, and the schema's resource enum makes the target explicit. It communicates the key capability areas (fields, filters, sorting, paging) but does not explicitly contrast itself with sibling tools like jobboss2_get or jobboss2_describe_resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this list tool versus the get, describe_resource, or report siblings. The description implies broad list usage but provides no exclusions, prerequisites, or alternative routing, leaving the agent to infer when this tool is the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_createA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds the useful behavioral detail that request data is validated against ECI's OpenAPI schema, which is beyond the annotations. It does not describe side effects, idempotency, or failure modes, but the annotations already signal readOnlyHint=false and idempotentHint=false without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The purpose is front-loaded, and the second sentence gives actionable workflow guidance that earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a clear entry point for a complex creation tool by routing the agent to jobboss2_describe_resource for per-resource details. With an output schema present and annotations covering safety traits, the remaining gap is not explaining the distinction from create_report or the exact role of parentKeys.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate by explaining data, resource, and parentKeys. It only mentions 'required parent keys and fields' and defers to another tool for details; it does not explain the data parameter or how resource selects the record type. The resource enum helps, but the core data object remains underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and the resource ('a JobBOSS2 record'), and the mention of ECI's OpenAPI schema adds specificity. It is distinguishable from the get/list/update/describe siblings, though it does not explicitly distinguish itself from jobboss2_create_report beyond the word 'record'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit prerequisite: use jobboss2_describe_resource first to inspect required parent keys and fields. This tells the agent how to prepare a valid call, but it does not explicitly contrast with alternatives such as update or create_report.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_issuing_materials_lookupA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint, covering the safety profile. The description adds that the tool supports 'standard JobBOSS2 filters and paging', but nothing beyond that; no side effects, auth requirements, or edge-case behavior are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence with no filler. It front-loads the core purpose and keeps the supporting details about filters and paging compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only lookup tool with annotations and an output schema, the description covers the basic purpose. Still, the lack of parameter semantics and explicit usage guidance means an agent would likely need to inspect the schema closely before constructing a valid invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only gestures at 'standard JobBOSS2 filters and paging' without explaining skip, take, sort, fields, or filters. It does map the lookup enum to human-readable categories (bin locations, job numbers, routing steps), but this is insufficient for a 6-parameter tool with no schema-level descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('List') and concrete resources: 'issuing-material bin locations, job numbers, or routing steps'. It clearly distinguishes this lookup tool from generic siblings like jobboss2_list and jobboss2_get.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: when the agent needs issuing-material reference data such as bin locations, job numbers, or routing steps. However, it does not explicitly state when not to use it, nor does it name alternatives like jobboss2_get or jobboss2_transfer_stock_to_job.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_transfer_stock_to_jobA
  3.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructive and non-idempotent behavior, but the description adds concrete detail: it changes bin quantities and can create job materials, inventory transactions, and job requirements. It also explicitly warns that retrying is unsafe, which is critical operational context beyond the annotations. No contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences - the purpose first, then effects, then a safety warning. Every sentence adds distinct information: what it does, what changes, and caution about retries. There is no fluff or redundancy, and the most important operational constraint (retry safety) is front-loaded after the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a mutation tool with a single, opaque 'data' parameter and no documentation of its structure. The description covers side effects and retry risk, but it omits required inputs, expected format, or any clues about what constitutes valid data. Without that, an agent cannot reliably invoke the tool, making the description inadequate for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage, and the only parameter 'data' is an untyped, recursive object with no field documentation. The description does not explain what data should contain (e.g., job ID, stock details, quantities). With such low schema coverage, the description must compensate, but it provides no parameter guidance whatsoever, leaving an agent unable to construct a valid request.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Transfer stock material to a job.' This clearly defines the operation and distinguishes it from siblings like 'update' or 'create' by focusing on the transfer-action semantics. The title and description align, leaving no ambiguity about the tool's purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case (transferring stock to a job) but does not explicitly contrast with sibling tools or state when not to use it. There is no mention of alternatives or prerequisites, so an agent must infer from the name that this is the tool for stock transfers. It provides no conditions for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_get_reportA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description warns that the response schema is not defined by the current OpenAPI document, which is important behavioral context beyond the readOnlyHint and idempotentHint annotations. It also reinforces the read-only nature by describing retrieval of an existing report. It does not detail error behavior or auth requirements, but the annotation coverage plus the schema caveat puts this above the minimum.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The definition is two short sentences: the first states the action and key parameter, the second flags a schema limitation. There is no repetition, filler, or unnecessary detail. Every sentence contributes useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read-only tool with idempotence and open-world annotations, the description covers the essential behavior and even notes the undocumented response schema. A small gap is not connecting requestId to the create_report workflow, which would help an agent chain calls correctly. Overall, it is nearly complete for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description provides the only semantic weight for requestId by explaining that the report is retrieved 'by request ID.' This is helpful but minimal; it does not say where the request ID comes from or how to format it. The parameter name itself already suggests its role, so the added value is modest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb and resource: 'Get a previously submitted JobBOSS2 report by request ID.' It distinguishes this from report creation by emphasizing 'previously submitted,' and the name itself separates it from the generic jobboss2_get sibling. It could be stronger with an explicit contrast to create_report, but the core purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'previously submitted' implies this tool is for retrieving already-created reports, which gives the agent a usable context cue. However, it does not explicitly say to use create_report first, nor does it state when not to use this tool compared to jobboss2_get or other siblings. The usage guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_describe_resourceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds that the tool returns paths, keys, query parameters, and request schema, which is useful but not deep behavioral disclosure such as resource naming/pagination or operation-specific behavior. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The main capability is front-loaded, and the usage directive is preserved for the second sentence. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an introspection tool with an output schema and read-only annotations, the description covers the core capability, what the agent can inspect, and when to call it. The only minor gap is that it does not spell out how the optional operation parameter changes the behavior between listing resources and inspecting one resource.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It implies the 'resource' parameter selects which resource to inspect and that 'operation' relates to the intended create/update/get/list action through the phrase 'before create or update tools.' However, it does not explicitly explain the operation parameter's behavior or default when omitted, leaving some burden on the enum values in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as an introspection/metadata tool: it 'List JobBOSS2 resources or inspect one resource's paths, keys, query parameters, and request schema.' This distinguishes it from the actual data-operation siblings (create, update, list, get) and explains exactly what the agent will learn by calling it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit guidance on when to call it: 'Call this before create or update tools.' This is a clear usage context. It does not explicitly list exclusions for other siblings, but the purpose is distinct enough that the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_updateA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal mutation (readOnlyHint=false, destructiveHint=true), and the description adds meaningful behavior: request data is validated against ECI's OpenAPI schema. It also flags that key names and writable fields may need discovery, but doesn't detail post-update effects or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences, with the core action first and the required discovery step immediately after. No filler or duplication of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema, annotations, and the explicit describe-first instruction, the agent has enough to attempt a correct call. The only gap is a direct explanation of the `keys` parameter's role, though the description points to a tool that resolves it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description carries the burden. It indirectly explains that `keys` identifies key names and `data` contains writable fields, but never explicitly defines the three parameters or how they combine. The resource enum is self-explanatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a concrete action ('Patch') on a JobBOSS2 record, which is clear and update-specific. It doesn't explicitly contrast with jobboss2_create or jobboss2_get, though the PATCH verb implies modifying an existing record.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a clear prerequisite: run jobboss2_describe_resource first to inspect key names and writable fields. It does not discuss when to choose update over create/list, so it stops short of a full routing guide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jobboss2_getA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already convey readOnly and idempotent behavior. The description adds useful context beyond that: the required key names are resource-specific and must be looked up, which is a meaningful behavioral nuance for calling this tool correctly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The core action is front-loaded and the lookup hint earns its place as essential usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the readOnly and idempotent annotations, an output schema, and the resource enum in the input schema, the description covers the main ambiguity: mutable resource-specific key names. It could still mention the 'fields' parameter, but the overall tool behavior is adequately contextualized.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description has the burden of explaining parameters. It partially compensates by explaining that 'keys' must contain the exact key names for the chosen resource. However, it does not explain the 'fields' parameter or clarify its role in selecting returned data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description says 'Get one JobBOSS2 record', which names a specific verb, resource, and singular scope. This clearly distinguishes it from jobboss2_list and jobboss2_get_report without needing to inspect the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives an explicit practical instruction: use jobboss2_describe_resource to find the exact key names required by the resource. The 'one record' phrasing implies when to choose this over list, though it does not explicitly state exclusions or name the alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.