Server Quality Checklist

Tool Scores

• health_checkB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that this is a verification/reporting operation (non-mutating), and partially explains behavior: checks endpoint reachability and 'expected top-level shape' for recipes, and 'runs through the ladder' for URLs. However, 'the ladder' is undefined jargon, and the odd parenthetical about 'known-good test slug' is confusing and doesn't clarify what a failed check actually does.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description front-loads the purpose well, but the parenthetical about 'known-good test slug isn't available generically' is confusing, self-referential, and nearly unparseable. This sentence should be rewritten or removed since it undermines clarity without adding actionable value. The description is somewhat rambling and could be tightened.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is an output schema present, so return-value details are covered structurally. The tool has only 1 parameter which is reasonably explained, but the parenthetical confusion and the undefined 'ladder' concept leave notable gaps. Given the absent annotations and moderate complexity of dual-target behavior, the description should be more explicit about what constitutes a healthy versus broken result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is only 1 parameter (target) with 0% schema description coverage, so the description must compensate. It does explain that target accepts either a recipe id or a plain URL and differentiates the behavior for each. However, it doesn't describe the expected format of the target (how to distinguish a recipe id from a URL, any prefix/ID format), leaving some ambiguity for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: verify a recipe or raw URL is still working, and distinguishes it from the silent-failure alternative of discovering breakage only after a real search. It names the two input forms (recipe id or plain URL) and what each checks. However, it doesn't explicitly name sibling tools like fetch or use_recipe to differentiate, and the purpose is somewhat muddled by the parenthetical about 'known-good test slug isn't available generically'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys when to use it (proactively verify before discovering breakage), and distinguishes recipe-id checks from URL checks. However, it doesn't explicitly state when NOT to use it or name alternative tools (fetch, use_recipe, extract) to clarify boundaries. The usage guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• use_recipeA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description gives useful hints (raw API responses often don't include entity name, hence entity_name param) but doesn't disclose side effects, whether it writes to a system, rate limits, auth requirements, what gets persisted/attached, or what happens on invocation—whether results are stored or merely returned. For an 'invoke' tool with zero annotation coverage, this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-organized, with each parameter getting its own brief docstring line with concrete examples. The prose is efficient with minimal waste. It could be slightly more front-loaded about the overall action, but the docstring-style format is clean and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is an output schema present, which partially reduces the burden on the description for return-value explanation. The description explains the three parameters well and covers the invocation semantics. However, it omits behavioral context that matters for a recipe invocation: does it write/update data, is it safe/idempotent, are there prerequisites beyond a valid recipe_id? With no annotations, these gaps make the description feel incomplete for a tool that evidently has side effects worth understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It does well: recipe_id gets concrete examples (ats_greenhouse, ats_lever, ats_ashby), slug gets a Greenhouse board token example with the boards.greenhouse.io/<slug> URL pattern, and entity_name gets its rationale (results often lack company name) plus an example. This adds meaningful value well beyond the bare schema field titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: invoke a specific recipe by id against one entity's slug/identifier on that platform. The verb 'invoke' plus the resource (recipe by id) is specific. It distinguishes somewhat from siblings by referencing list_recipes as the source for recipe ids, though it doesn't explicitly contrast with fetch/extract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description references list_recipes as the source for recipe ids, giving implicit guidance on how to obtain valid recipe_id values. It provides clear contextual use (invoking recipes against a platform entity). However, it doesn't explicitly state when NOT to use this tool or name specific alternatives, though the sibling context (fetch/extract/list_recipes) makes the distinction reasonably inferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extractA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries full behavioral burden and does well: it discloses that the tool uses an LLM (hence slower/less deterministic), requires ANTHROPIC_API_KEY, and specifies failure behavior (diagnosable failure, not crash). This is meaningful operational context beyond the schema. It could note potential cost/latency implications but already covers the key operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a purpose paragraph followed by an Args section. The json_schema example is slightly verbose but earns its place since it demonstrates the exact expected format. One minor inefficiency: the API-key disclosure could be more concise, but the overall structure is clean and front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is moderately complex (LLM-based, 3 params including a nested JSON schema string), and despite no annotations, the description covers purpose, fallback behavior, an example, auth requirement, and failure mode. The output schema is present, so return-value documentation isn't required. A note on typical latency or cost of LLM extraction would be a nice addition, but the core operational needs are satisfied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all three parameters. It does: url (URL to extract from), json_schema (JSON Schema string with a concrete example), and instructions (optional guidance). The json_schema example is particularly valuable since the schema's title 'Json Schema' alone is insufficient for an agent to construct a valid value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (fetch+extract), the resource (URL with structured fields), and the mechanism (LLM-based, works on ANY site). It explicitly distinguishes itself from the sibling `list_recipes`/`use_recipe` tools, noting the alternative is for sites with dedicated deterministic recipes. This is specific and well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit when-to-use directive: use `list_recipes` first if a deterministic recipe may exist, otherwise use this tool for generic extraction. This directly addresses the usage decision relative to its sibling tools and provides clear guidance on the alternative path.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetchA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It does well here: discloses the return shape (confidence, method, tier), instructs the agent to check confidence before trusting, and guarantees no silent empty results on failure (failure_reason). This is rich, actionable behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear one-line summary, then formatted Args section, then return/failure details. Every sentence earns its place — no fluff, front-loaded with the key purpose statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having only 2 simple params and an output schema, this is a complex tool (multi-tier escalation). The description covers the fetch strategy, both parameters, the return contract, confidence guidance, and failure semantics. Exceptionally complete for its complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description must compensate fully. It documents both parameters: url (clear from name/schema) and mode, explaining markdown vs raw semantics with clear defaults. The mode explanation adds real value beyond the schema's bare type/default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource ("Fetch a URL") with specific mechanism described (resilience ladder with tier escalation). Distinguishes itself from sibling extract tool by being the URL-fetching primitive, and references a distinctive multi-tier strategy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Describes the escalation behavior (static -> rendered, escalating when failure suggests it helps), giving an agent context on how the tool self-optimizes. Doesn't explicitly name alternatives or say when NOT to use it, but the clear scope of the tool makes this a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recipesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It conveys that the tool is a read-only listing operation (implicitly safe, non-destructive) and frames recipes as 'verified parsers' with a speed/confidence advantage. It doesn't explicitly state return format, but the presence of an output schema partially covers return-value disclosure. For a simple list op with no annotations, this is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences deliver complete value with zero waste. The first sentence states the purpose with concrete examples, and the second provides actionable usage guidance that ties into a sibling tool (extract). Every clause earns its place; no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is parameterless with a clean purpose and an output schema present, the description is complete. It explains what the tool returns (registered extraction recipes) and how to use it in the workflow (before extract with manual schema). The presence of an output schema relieves the description from detailing return structure. Nothing meaningful is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  This tool has 0 parameters, so per the rubric the baseline is 4. There is no schema info to add value beyond, and the tool genuinely doesn't need parameters. The description focuses entirely on output semantics rather than inputs, which is appropriate for a parameterless list operation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the specific purpose: 'List every registered extraction recipe (platform-specific, verified parsers — e.g. Greenhouse, Lever, Ashby job boards).' The verb 'List' clearly identifies the action and the resource (extraction recipes) is precisely scoped. It distinguishes from siblings by naming the resource type and examples, making it unambiguous what this tool returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: 'Check this before calling `extract` with a manual schema: a matching recipe is faster and higher-confidence than LLM extraction.' This tells the agent exactly when to use this tool (before extract) and explains the rationale, effectively framing it as a prerequisite decision step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.