Server Quality Checklist

Tool Scores

• search_jobsC
  2.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is known. The description adds 'Descriptions are sanitized and never interpreted,' which gives some insight into how the search operates (likely on sanitized fields, ignoring untrusted content). This adds modest behavioral context beyond the annotations, though it remains vague about what 'interpreted' means.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, very concise, with the main purpose front-loaded. There is no filler or repetition. However, the second sentence about sanitization might be extraneous for purpose clarity but adds behavioral transparency. Structurally it's efficient, though it sacrifices completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters with no schema descriptions and a search operation with potential nuances, the description is inadequate. It lacks parameter explanations, usage examples, or any mention of how to constrain the search. The output schema exists, so return values are covered, but the lack of parameter semantics and usage context makes it incomplete for an agent to call correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must explain the parameters (query, limit, remote_only). The description fails to mention any of them. An agent cannot infer what 'query' refers to, what 'limit' caps, or what 'remote_only' filters. This is a severe gap given the schema provides no descriptions either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Search trusted listing fields' – a clear verb (search) and resource (trusted listing fields). It distinguishes itself from siblings by being a search operation, though it doesn't explicitly name alternatives. The phrase 'trusted listing fields' is somewhat vague about what fields are included, but it's still a specific purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given as to when to use this tool versus siblings like rank_jobs or export_shortlist. The description does not mention alternatives or provide context about choosing this over others. There is no 'when to use' or 'when not to use' information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_matchB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description does not need to repeat safety traits. It adds context that the explanation includes supporting signals and tradeoffs, which is useful but not essential. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is exactly as long as needed, with no filler or redundant phrasing. The core purpose is front-loaded and the sentence is perfectly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a complex profile parameter and an output schema, but the description does not explain what profile is for or any prerequisites (e.g., job must exist). While the output schema covers return format, the missing context around the required profile and the implied validity of job_id makes this incomplete for an agent to call correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It only hints that job_id identifies a known job but does not explain the 'profile' object at all, and it provides no details on fields like skills, seniority, or salary. The description fails to add meaning beyond the schema's bare structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Explain') and a specific resource ('the score, supporting signals, and tradeoffs') for one known job, clearly differentiating from siblings like search_jobs (which finds jobs) and rank_jobs (which ranks multiple). It is unambiguous and directly tied to the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: one uses this when they have a specific job_id and want an explanation of its match. However, there is no explicit guidance on when to prefer this over rank_jobs or search_jobs, and no exclusions or alternatives are mentioned. The context is clear but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_shortlistA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is known. The description adds specific behavioral context by stating it never writes files or applies to jobs, which clarifies exactly what side effects are absent. This goes beyond a generic read-only hint, giving the agent confidence about the tool's non-persistent nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences: the first states the purpose, the second clarifies boundaries. Every word earns its place, and the key action is front-loaded. No unnecessary detail or repetition of schema or annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, annotations cover safety, and an output schema exists. The description provides the essential behavioral boundary (no file writes, no job application). The main gap is lack of explicit guidance on when to use this tool relative to siblings, but for a simple export tool with self-explanatory parameters, the information is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema_description_coverage is 0%, so the description must compensate for missing parameter explanations. It does not mention format or job_ids at all. The schema shows format as an enum and job_ids as an array, but their purpose is only implicitly tied to 'render a shortlist.' This is insufficient compensation for zero schema descriptions, though the parameters are simple enough that some inference is possible.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Render a shortlist for the client') with a resource ('shortlist'). It also differentiates from side-effectful tools by noting it never writes files or applies to jobs, which helps distinguish it from potential sibling operations. However, 'render' is slightly ambiguous (display vs generate), and it doesn't explicitly contrast with sibling tools like search_jobs or rank_jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a shortlist needs to be presented to the client, and the phrase 'never writes files or applies to jobs' provides some exclusionary guidance. However, it doesn't explicitly state when to use this tool over alternatives or mention sequencing (e.g., after rank_jobs). The context is present but not fully articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rank_jobsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds the behavioral trait of a 'deterministic, transparent scoring policy,' which informs the agent that rankings are reproducible and likely explainable, adding context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose ('Rank known jobs') and appends the key policy detail. No words are wasted, and every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has three parameters including a richly defined CandidateProfile object, the description is minimal. It does not explain how the profile is used, the effect of limit, or the nature of the scoring policy in practice. The output schema exists, which helps, but the description alone leaves notable gaps for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention any parameters or how the profile affects scoring. The schema provides titles and types, but the description fails to compensate for the low coverage, leaving the role of job_ids and the complex CandidateProfile fields underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Rank known jobs' specifies a clear verb and resource, and the qualifier 'deterministic, transparent scoring policy' adds unique value that distinguishes it from siblings like search_jobs (searching), export_shortlist (exporting), and explain_match (explaining). It clearly indicates the tool ranks a predefined set of jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'known jobs' provides clear context that this tool operates on job IDs already obtained, aligning with search_jobs as a precursor. It does not explicitly state when not to use it, but the context is clear enough to route an agent to the appropriate tool among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.