Server Quality Checklist

Tool Scores

• create_labelling_requestB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions triggering validation and trial setup, indicating side effects, but does not specify if the operation is destructive, what happens on failure, or any required preconditions. Lack of detail on mutation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that effectively communicates the core purpose. It is front-loaded and wastes no words. Could include more detail on usage or outcome without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (nested object, no output schema, no annotations), the description is too minimal. It does not explain return values, error handling, typical usage flow, or integration with other tools. Missing important context for a complex submission tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100% with a single parameter. The schema description 'Comprehensive hierarchical clinical labelling structured data payload' is already provided. The tool description adds no additional semantic insight beyond what the schema offers. Baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (submits), the resource (Labelling Design Request payload), the destination (Appian), and the outcome (trigger validation and trial setup). It distinguishes from siblings like create_procurement_request which is for procurement, and get_appian_suppliers which is for suppliers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when the request is 'completely populated' and 'multi-step,' but does not explicitly state when to use it vs alternatives, nor does it mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_procurement_requestB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries full burden. It states that the tool starts a workflow, but lacks details on side effects (e.g., whether it is idempotent, permission requirements, or error behavior). This is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-formed sentence that conveys the core action without any redundant words. It is appropriately front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a nested parameter and no output schema, the description should clarify what the tool returns (e.g., a request ID) or the outcome after submission. It only describes the input and action, leaving the return behavior unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds that the payload is 'completely structured' and 'multi-step', but does not provide further meaning beyond the schema's object definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'submits' and the resource 'a completely structured multi-step procurement form payload', with the target 'into Appian to start the workflow process'. It distinguishes from siblings like create_labelling_request, which likely handles a different form type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a procurement request needs to be created, but provides no explicit guidance on when to use this tool versus alternatives (e.g., create_labelling_request), nor does it mention prerequisites or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recordsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry full behavioral disclosure. However, it only states the basic action without mentioning authentication, rate limits, error handling, or what happens if the record type doesn't exist. The pageSize parameter hints at pagination but behavior is not described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the tool's purpose. It is appropriately sized and front-loaded with the key action and resource. Minor room for improvement by adding a brief note on return format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 parameters, no output schema), the description is minimally adequate. It tells what the tool does and the key parameter. However, it lacks details about return values, error handling, and recordName format, which are gaps for contextual completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters (recordName, pageSize) are documented in the schema. The description adds no additional meaning beyond the schema, which meets the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve records' with the resource 'records from an Appian application' and the method 'using record type name'. It distinguishes from sibling tools like create_labelling_request and get_appian_suppliers, which focus on creating or getting specific entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives. While the purpose implies retrieval, there is no guidance on when not to use it or mention of specific contexts (e.g., filtering). The usage is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_appian_suppliersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool performs a read operation (fetch), but lacks details on pagination, authentication, rate limits, or return format. The description is minimally adequate for a simple read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 16 words, front-loaded with the verb and object. Every word is necessary and no extraneous information is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple nature (one required parameter, no output schema), the description covers the core purpose and parameter. It lacks behavioral details and return structure, but remains fairly complete for a straightforward fetch tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the parameter 'category' described as 'The business category filter.' The description adds 'based on a given domain/category' which is redundant. It does not significantly enhance understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'fetches', the resource 'suppliers', the source 'Appian Data Fabric', and the condition 'active available'. It also specifies filtering by domain/category, distinguishing it from sibling tools like create_labelling_request or get_records.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when needing a list of active available suppliers filtered by category. However, it does not explicitly mention when not to use it or suggest alternative tools, though the context is sufficient for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.