Deterministically extract failure signal from a job's log or a JUnit/Surefire artifact

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the operation reads a log or artifact and uses 'real parsers (JUnit/Surefire XML, ESLint stylish, tsc, Prettier, Maven console), not an LLM guessing from raw text.' This adds valuable behavioral context beyond a simple 'extract' verb, implying deterministic, read-only behavior. It does not discuss edge cases or errors, but the read-only nature is clear through 'reads' and 'extract.'

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences that are information-dense: the first defines the transformation and output, the second provides usage guidance. There is no filler or repetition of schema field names. It is well-structured and front-loaded with the primary purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema and no annotations, so the description must cover return semantics. It does list the output fields (test name, class, message, stack trace, signature), which is helpful. However, it does not explain behaviors when no failure signal is found, how errors are handled, or whether the output is always a list. It also doesn't address potential edge cases like missing artifacts or malformed logs, leaving some ambiguity for a parser tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 80% (4 of 5 parameters have descriptions). The description adds meaning for artifactName by explaining it selects structured XML instead of the job log. However, runId lacks any description in either the schema or the description, and the description does not compensate for that gap. Overall, it meets the baseline but doesn't significantly elevate parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Turns a noisy raw CI log or a downloaded JUnit/Surefire XML artifact into structured FailureSignal entries') with a defined resource (the log or artifact) and output (test name, class, message, stack trace, signature). It distinguishes from siblings by focusing on extraction/parsing rather than fetching, searching, recording, or triaging.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit usage guidance: 'Prefer passing artifactName when fetch_pipeline_run listed one matching /junit|surefire/i for this run; otherwise this reads the job's own console log.' This provides a clear conditional for when to use the artifact vs the log. It does not explicitly name alternative sibling tools for the same task, but the conditional context is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.