Server Quality Checklist

Tool Scores

• get_job_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves status but doesn't mention whether this is a read-only operation, if it requires specific permissions, potential rate limits, error conditions (e.g., invalid ID), or the response format (e.g., JSON with status field). This leaves significant gaps for a tool that likely queries system state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get the current status') and includes helpful examples. There's no wasted verbiage, repetition, or unnecessary elaboration, making it appropriately sized for a simple lookup tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that interacts with job executions. It doesn't explain what the return value includes (e.g., status enum, timestamps, error details), how to interpret statuses, or error handling, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'jobExecutionId' fully documented in the schema. The description doesn't add any parameter-specific details beyond what the schema provides (e.g., format examples, ID sources), so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'current status of a job execution', with specific examples of possible statuses (IN PROGRESS, COMPLETED, FAILED). However, it doesn't explicitly differentiate from sibling tools like 'get_job_results' or 'get_job_audit_log', which likely provide different job-related information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a jobExecutionId from 'execute_job' or 'initiate_job'), nor does it clarify what distinguishes it from siblings like 'get_job_results' (which might return output data) or 'get_job_audit_log' (which might provide logs).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• initiate_jobC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool initiates a job and returns an ID, but lacks details on permissions, side effects (e.g., if it starts execution immediately), error handling, or rate limits. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence and adds a critical detail about the return value in the second. Every sentence earns its place with zero waste, making it appropriately sized and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of initiating a job (a mutation operation) with no annotations and no output schema, the description is incomplete. It doesn't explain what 'initiate' entails (e.g., does it start execution?), potential side effects, or error scenarios, leaving gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters (workflowId, title, description). The description adds no additional meaning beyond what the schema provides, such as format examples or constraints, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Initiate a new job') and resource ('for a workflow'), distinguishing it from siblings like 'get_job_status' or 'execute_job'. However, it doesn't explicitly differentiate from 'execute_job' which might be a similar action, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'execute_job' or 'get_workflow_details'. It mentions that the returned 'jobExecutionId' is required for subsequent operations, but this is more of a consequence than usage guidance, lacking explicit when/when-not instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_jobB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that inputs should be 'populated according to workflow schema' but fails to disclose critical traits such as whether this is a read-only or destructive operation, authentication needs, rate limits, or what happens upon execution (e.g., job status changes). This leaves significant gaps for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two concise sentences that are front-loaded with the core purpose. There is no wasted text, and it efficiently conveys necessary information without being overly verbose, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing a job with nested parameters and no output schema or annotations, the description is incomplete. It lacks details on behavioral traits, error handling, return values, or how this tool interacts with siblings like 'initiate_job'. For a tool that likely triggers significant system actions, this leaves too many unknowns for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already documents both parameters thoroughly. The description adds minimal value by referencing 'jobPayloadSchema from get_workflow_details' for structuring the 'jobPayloadSchemaInstance', but this is redundant with the schema's description. No additional semantic context is provided beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute a job') and specifies the resource ('with populated input values'), making the purpose understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'initiate_job' or 'get_job_status', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage by referencing 'jobPayloadSchema from get_workflow_details' for structuring inputs, suggesting a dependency. However, it lacks explicit guidance on when to use this tool versus alternatives like 'initiate_job' or 'get_job_results', and doesn't mention any exclusions or prerequisites beyond the schema reference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_job_audit_logB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves audit logs but doesn't mention whether this is a read-only operation, if it requires specific permissions, what format the logs are in, or if there are rate limits. This leaves significant gaps for a tool that likely accesses sensitive execution data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple retrieval tool and front-loads the key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one well-documented parameter but no annotations or output schema, the description is minimally adequate. It explains what the tool does but lacks important context about behavioral traits, return format, and how it differs from sibling tools, leaving the agent with incomplete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting the single required parameter. The description adds no additional parameter semantics beyond what the schema provides, but since schema coverage is high, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('detailed audit log of all system actions during job execution'), making the tool's function understandable. It doesn't explicitly differentiate from sibling tools like get_job_status or get_job_results, but the focus on audit logs provides some implicit distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_job_status or get_job_results. It mentions 'during job execution' but doesn't clarify prerequisites, timing constraints, or exclusions, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_workflow_detailsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a 'Get' operation, implying it's read-only, but doesn't confirm safety aspects like whether it requires authentication, has rate limits, or what happens if the workflowId is invalid. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose. It's front-loaded with the core action and includes a key detail (jobPayloadSchema) without unnecessary elaboration. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and hints at the return content (jobPayloadSchema), but lacks context on usage, behavioral traits, or output structure. Without annotations or output schema, the agent has incomplete information for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'workflowId' fully documented in the schema. The description adds no additional parameter details beyond what the schema provides (e.g., format examples or constraints). According to the rules, with high schema coverage, the baseline is 3 even without param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'workflow details', specifying that it retrieves information including the jobPayloadSchema. It distinguishes itself from siblings like execute_job or get_job_status by focusing on workflow metadata rather than job execution or status. However, it doesn't explicitly contrast with all siblings (e.g., get_job_audit_log).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a workflowId), exclusions, or comparisons to siblings like get_job_results or initiate_job. The agent must infer usage from the tool name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_job_resultsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It adds useful context by stating the prerequisite that the job must be COMPLETED, which isn't obvious from the schema alone. However, it lacks details on error handling, return format, or potential side effects, leaving gaps in behavioral understanding for a tool that retrieves data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of just one sentence that directly states the tool's function and key usage condition. Every word earns its place, with no redundant or unnecessary information, making it efficient and easy to understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (retrieving results based on a status condition), no annotations, and no output schema, the description is somewhat complete but has gaps. It covers the core purpose and a critical usage constraint, but lacks details on return values, error cases, or how results are structured, which could hinder an agent's ability to use it effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'jobExecutionId' fully documented in the schema. The description doesn't add any additional meaning or details about the parameter beyond what the schema provides, such as format examples or constraints. This meets the baseline score of 3 since the schema handles the parameter documentation effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving results for completed job executions. It specifies the verb 'get' and the resource 'results of a completed job execution,' making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_job_status' or 'get_job_audit_log,' which might handle related but different data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: only when a job's status is COMPLETED. This implies that alternatives like 'get_job_status' should be used to check status first. However, it doesn't explicitly name these alternatives or detail when not to use this tool beyond the status requirement, such as for ongoing or failed jobs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_file_upload_urlA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns two URLs (presignedUrl and fileUrl) and their purposes, which adds behavioral context. However, it lacks details on permissions needed, rate limits, expiration times for URLs, or error conditions, leaving gaps in behavioral transparency for a tool that generates upload URLs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence and efficiently adds return value details in the second sentence. There is no wasted text, and every sentence earns its place by providing essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and return values but lacks details on behavioral aspects like authentication, rate limits, or error handling. For a tool with 2 parameters and no structured output documentation, it is minimally adequate but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the baseline is 3. The description does not add specific details about parameters beyond what the schema provides, but it contextually implies that fileExtension is used to generate a URL for a file of that type, and accessScope might relate to who can access the uploaded file. This adds marginal semantic value, raising the score slightly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Generate presigned URL for file upload') and the resource ('file upload'), distinguishing it from sibling tools like execute_job or get_job_status that handle job execution and monitoring. It explicitly mentions what the tool returns, which helps differentiate its purpose from other tools in the server.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning that the generated fileUrl is 'to reference in job execution', suggesting this tool is used as a preparatory step for jobs. However, it does not explicitly state when to use this tool versus alternatives or provide any exclusions or prerequisites for its use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.