Server Quality Checklist

Tool Scores

• get_build_statusB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden of behavioral disclosure. It states this is a read operation ('Get'), but doesn't describe what 'Build information dictionary' contains, whether there are rate limits, authentication requirements, error conditions, or how 'latest' is determined when build_number is null. The return format is mentioned but not detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise: purpose statement followed by Args and Returns sections. Every sentence earns its place - no redundant information, well-organized, and front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter read tool with no annotations and no output schema, the description provides basic but incomplete context. It covers the purpose and parameters adequately, but lacks details about the return value format, error handling, system context, or integration with sibling tools. The absence of output schema means the description should ideally explain the return structure more thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clearly explains both parameters: 'job_name' as 'Name of the job' and 'build_number' with its default behavior ('defaults to latest'). This adds meaningful context beyond the bare schema, though it doesn't specify format constraints or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with 'Get build status' - a specific verb ('Get') and resource ('build status'). It distinguishes from siblings like 'list_jobs' (which lists jobs) and 'trigger_build' (which initiates builds). However, it doesn't explicitly mention what system or context these builds belong to (e.g., CI/CD pipeline).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of when to use 'get_build_status' instead of 'list_jobs' (which might provide status overview) or 'trigger_build' (which creates new builds). No context about prerequisites, timing, or workflow integration is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_jobsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states 'List all Jenkins jobs' but doesn't disclose behavioral traits such as pagination, rate limits, authentication needs, or what 'all' entails (e.g., scope, filtering). This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded and appropriately sized for a simple tool, earning full marks for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, output schema exists), the description is adequate but incomplete. It lacks behavioral context (e.g., how jobs are returned, any limitations), which is needed since no annotations are provided. The output schema helps, but the description should add more value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter information is needed. The description doesn't add param details, but that's unnecessary here, meeting the baseline for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all Jenkins jobs'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_build_status' or 'trigger_build', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_build_status' or 'trigger_build'. It lacks context about prerequisites, timing, or exclusions, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trigger_buildB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the action ('Trigger a Jenkins build') and return format, but doesn't cover important aspects like authentication requirements, rate limits, whether this is idempotent, what happens if the job doesn't exist, or potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with clear sections (Args, Returns) and uses minimal words to convey essential information. Every sentence serves a purpose without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description provides basic purpose and parameter information but lacks important behavioral context. It covers what the tool does and what parameters mean, but doesn't address authentication, error handling, or system impact sufficiently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the schema, which has 0% description coverage. It explains that 'job_name' identifies the job to build and 'parameters' are optional build parameters with a helpful example. This compensates well for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Trigger') and resource ('Jenkins build'), making the purpose immediately understandable. However, it doesn't explicitly differentiate this tool from its siblings (get_build_status, list_jobs) beyond the obvious action difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives like get_build_status or list_jobs. The description lacks context about prerequisites (e.g., job must exist), timing considerations, or error conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.