Server Quality Checklist

Tool Scores

• compare_companiesA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses a key behavior: partial failure isolation ('One company failing does not lose the others; each failure is reported beside the results that did work') and normalization of field names across systems ('Companies on different systems come back with the same field names'). This provides valuable behavioral context beyond the schema. It does not cover every possible behavior (e.g., auth, rate limits), but the core transparency is commendable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat verbose and literary, with phrases like 'This is the thing no single applicant-tracking system can do' and 'each vendor only knows about its own customers' which add flavor but no operational value. The core information (multi-company, normalized fields, failure isolation) is present but could be stated more tersely. It's not overly long, but it has noticeable fluff, so a mid score is warranted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description should explain what the agent can expect as a result. It mentions that field names are unified and failures are reported side-by-side, but it does not describe the actual return structure (e.g., array of company results, status codes, error format). For a tool with two parameters and no output schema, this is a moderate gap. The tool is otherwise reasonably well-specified, so a 3 is fitting.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% — both 'companies' and 'per_company' already have descriptions in the schema. The tool description does not add any additional meaning or syntax details beyond what the schema provides (e.g., it doesn't clarify URL formats or constraints further). With full coverage, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb+resource: 'Open jobs across several companies at once, in one schema.' It explicitly differentiates from single-company ATS tools by highlighting cross-system comparison, which distinguishes it from siblings like list_open_jobs or find_company_board. The purpose is unambiguous and not just a restatement of the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for multi-company comparison ('Open jobs across several companies at once') and notes it's 'the thing no single applicant-tracking system can do,' which suggests when to use it. However, it does not explicitly name alternatives or state when not to use it (e.g., for a single company, use list_open_jobs). The guidance is implied, not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_company_boardA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explains the underlying problem (APIs require board names, no directory exists) but does not describe the tool's own behavior—such as return format, error handling, or ambiguity handling. While context is provided, explicit tool behavior is missing, so a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The purpose is front-loaded in the first sentence, and the following paragraphs offer valuable domain context about why the tool is necessary and when to use it. The length is justified, though it could be slightly tightened without losing meaning. It reads naturally and every sentence contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters and no output schema, the description provides sufficient context: it explains the problem, the usage condition, and implies the return value (the ATS name). It lacks an explicit description of the return structure, but that's minor given the simple nature of the tool. A 4 reflects solid coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both 'company' and 'limit' already well-described in the schema. The description text adds no additional parameter-specific semantics beyond what the schema provides. Baseline 3 applies because the schema fully documents the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific, unambiguous action: 'Find which applicant-tracking system a company's careers page runs on.' This clearly distinguishes it from siblings like list_open_jobs by focusing on identification rather than job listing. It also adds context about the practical need for the tool, making its purpose obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit usage condition: 'Use this first when you have a company name rather than a careers URL.' This directs the agent on when to invoke it. However, it does not name alternative tools (e.g., compare_companies, list_open_jobs) or provide explicit 'when not to use' guidance, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_open_jobsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and meets it admirably. It discloses the refusal-over-guessing salary policy with specific edge cases (hourly rate, ambiguous currency, dual-currency bands yield no figure), that salary_text preserves the employer's raw text, that verify_url points to the employer's own advert, and (in the include_description schema field) that email addresses and phone numbers are stripped. These are non-obvious behaviors an agent needs to set expectations correctly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The purpose is front-loaded in the first sentence, and the prose is efficient. The salary paragraph is the longest but earns its space because the refusal behavior is genuinely non-obvious and critical to trust calibration. It is slightly verbose as three prose paragraphs where a scannable list would do, but every sentence carries information, so no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the tool is well covered: return fields are enumerated, salary refusal behavior is specified, company input formats live in the schema, and PII stripping is noted. The only gaps are pagination behavior (limit exists up to 1000 but continuation is undocumented) and error handling for unknown companies — minor omissions that prevent a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the schema's own parameter descriptions are rich (company accepts board name, careers URL, or system:name; include_description explains why it defaults off). The description adds complementary value by explaining what the tool returns for a given call, but it stays at the output/behavior level rather than clarifying parameter semantics directly — so it slightly exceeds the baseline-3 for full coverage without doing the schema's job.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening line states a precise verb and resource ('Every open job at one company, read from its own careers system') that immediately distinguishes it from find_company_board (locating a board) and compare_companies (comparing enterprises). Listing the exact return fields (title, department, location, employment type, salary range, verify_url) removes all ambiguity about what the tool yields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description firmly scopes usage to a single company ('Every open job at one company'), which is clear context, and the 'own careers system' plus verify_url language signals data provenance. However, it never names the sibling alternatives or states when NOT to use it (e.g., no explicit 'to compare companies, use compare_companies'), leaving the routing to inference rather than stating exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.