Server Quality Checklist

Tool Scores

• salary_contextA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the tool is powered by Adzuna and requires API keys, which is useful. However, it does not describe the output format, potential errors, or any rate limits. It adds some behavioral context but not comprehensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and includes the auth requirement. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description covers purpose, optional inputs, and auth requirements. It leaves some ambiguity about the exact distribution format, but it is mostly complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for both parameters (role and location), so the description adds no additional parameter meaning. Baseline 3 is appropriate given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a salary distribution (histogram) for a role and optional location. It uses a specific verb ('Return') and resource ('salary distribution'), distinguishing it from sibling tools like search_jobs or list_company_jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when salary data is needed for a role/location, but it does not explicitly state when to use this tool versus alternatives or provide any exclusions. Context is implied but not fully developed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_company_jobsA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions 'public postings' suggesting a safe read operation, but does not address pagination, authentication, rate limits, or response format beyond the implied list. This is insufficient for agents to anticipate side effects or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences efficiently state the core function and key parameter guidance. No redundancy, with the most important information front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main purpose and key parameters, but lacking an output schema, it does not specify the return format or pagination behavior. Optional parameters like query and minSeniority are only in the schema, and the description omits any mention of them, relying on the schema for their semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the schema by explaining 'provider' enum values and providing a concrete slug format example. It also clarifies that these are the two required parameters. Schema coverage is 75%, and the description fills the gap for the undocumented provider parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch all public postings from a single company's ATS job board,' specifying the verb, resource, and scope. It distinguishes from sibling search_jobs by emphasizing 'single company' and listing supported providers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this tool targets one company's ATS job board, implying when to use it (known company). However, it does not explicitly mention alternatives or when not to use it, such as for cross-company searches, which search_jobs likely handles.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_jobsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions optional sources (Adzuna/USAJobs) and filter capabilities, but does not explicitly state read-only behavior, rate limits, pagination, or how results are aggregated. The 'Search' verb implies a non-destructive operation, but without explicit disclosure it remains somewhat vague.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences with no filler. The first lists sources, the second outlines filter capabilities, and the third states the primary use case. Every sentence carries useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers sources, filters, and target use case. Since there is no output schema, it could have mentioned the return format or pagination, and it lacks a pointer to list_sources for source discovery. However, for a search tool with seven optional parameters, it provides sufficient orientation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description summarizes filter types (free-text, location, seniority, remote-only, date) and hints at minSeniority usage ('Ideal for scouting senior/leadership roles'), but adds no new semantic details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches job openings, explicitly enumerating the sources covered and mentioning it is ideal for senior/leadership roles. This distinguishes it from siblings like list_company_jobs (specific companies) and list_sources (source enumeration).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use when you want to search across multiple enabled sources, with an emphasis on senior roles. However, it does not explicitly mention alternatives or when not to use it, such as pointing to list_company_jobs for company-specific searches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sourcesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that this is a read-only listing of sources with status details, which is adequately transparent. It does not mention potential permissions or data sensitivity, but for a simple list tool this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action ('Show every source...') and immediately adds value with status and diagnostic context. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with no output schema, the description fully conveys what is returned (sources, enabled status, disabled reasons) and when to use it. It is complete and self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty (0 parameters). Per the rubric, the baseline is 4 when there are no parameters, and the description need not add parameter details. It correctly focuses on the output behavior instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool shows all job sources, their enabled status, and reasons for being disabled. This is a specific verb+resource combination that unambiguously distinguishes it from siblings like search_jobs or list_company_jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a clear context for when to use the tool (diagnosing configuration) but does not explicitly mention alternatives or exclusions. Still, the context is sufficient to guide appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.