Server Quality Checklist

Tool Scores

• get_jobC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves 'full details' but doesn't cover critical aspects like error handling (e.g., if the job ID is invalid), response format, authentication needs, rate limits, or whether it's a read-only operation. The description is minimal and leaves behavioral traits largely unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Get full details') and lists key data points. There's no wasted verbiage, and it's appropriately sized for a simple lookup tool. However, it could be slightly more structured by separating usage hints, but this is minor.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that retrieves data. It mentions what details are included but doesn't explain the return structure, potential errors, or dependencies. For a read operation with no structured output, more context on the response is needed to be fully helpful to an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'job_id' documented as 'Job handle or ID.' The description adds no additional meaning beyond this, such as format examples or sourcing hints. With high schema coverage, the baseline is 3, as the description doesn't compensate but also doesn't detract from the schema's clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('job listing'), listing key data fields like description, requirements, salary, and apply link. It distinguishes from sibling tools like 'search_jobs' (which likely returns multiple results) and 'find_similar_jobs' (which suggests broader matching), but doesn't explicitly contrast them. The purpose is unambiguous but could be more precise about differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a job ID), exclusions, or comparisons to siblings like 'search_jobs' (for broader queries) or 'find_similar_jobs' (for related listings). Usage is implied by the focus on 'specific job listing,' but explicit context is lacking.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_companyB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a 'Get' operation which implies read-only behavior, but doesn't specify whether it requires authentication, has rate limits, returns partial vs complete data, or handles errors. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get company profile') followed by specific data points included. Every word serves a purpose with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read operation with one parameter and no output schema, the description adequately covers what data is returned. However, without annotations or output schema, it doesn't specify the return format, error conditions, or data freshness. The description is minimally complete but leaves the agent to assume standard behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'company' documented as accepting domain or handle. The description doesn't add any parameter-specific information beyond what's in the schema, but doesn't need to since schema coverage is complete. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'company profile', specifying what information is included (open job count, industry, size, career page URL). It distinguishes from sibling tools like 'get_job' by focusing on company-level data rather than job-specific information, though it doesn't explicitly contrast with all siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_jobs' or 'find_similar_jobs'. It doesn't mention prerequisites, constraints, or typical use cases, leaving the agent to infer usage context from the tool name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_jobsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the dataset scale (1M+ jobs, 20K+ companies) and search capabilities, but lacks critical behavioral details: whether this is a read-only operation (implied but not stated), pagination behavior beyond parameters, rate limits, authentication requirements, error handling, or response format. For a search tool with 14 parameters, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: first establishes scope and scale, second enumerates capabilities. Every phrase adds value (scale numbers, search types, filter categories). Could be slightly more front-loaded by mentioning it's a search tool earlier, but overall well-sized without wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex search tool with 14 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what results look like (structure, fields returned), pagination strategy beyond parameters, sorting options, performance characteristics, or error scenarios. The agent must rely heavily on the input schema alone, missing critical context for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 79%, providing good baseline documentation. The description adds value by summarizing filter categories (location, salary, remote type, seniority, skills) and mentioning AI semantic search capability, which helps contextualize parameters like 'semantic_query'. However, it doesn't explain parameter interactions, default behaviors beyond schema defaults, or special syntax requirements beyond what's in schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches job listings with specific capabilities (keyword search, AI semantic search, filters). It distinguishes from siblings like 'find_similar_jobs' by emphasizing broad search across 1M+ listings rather than similarity matching, and from 'get_job' by focusing on search rather than retrieval of a specific job. However, it doesn't explicitly contrast with 'get_company', leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching job listings with various filters, but doesn't explicitly state when to use this tool versus alternatives like 'find_similar_jobs' (which likely finds similar jobs to a given one) or 'get_job' (which retrieves a specific job by ID). No guidance on prerequisites, error conditions, or performance considerations is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_similar_jobsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the method ('AI vector similarity') but lacks details on performance (e.g., speed, accuracy), limitations (e.g., data freshness, similarity thresholds), or output format (e.g., list structure, fields returned). For a tool with no annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence and adds a brief use case note in the second. Both sentences earn their place by providing essential information without redundancy or fluff, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (AI-based similarity search) and lack of annotations and output schema, the description is incomplete. It explains the purpose and method but misses behavioral details (e.g., what 'similar' means, response format) and usage nuances. This is adequate as a minimum viable description but has clear gaps for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (job_id and per_page) adequately. The description does not add any additional meaning beyond what the schema provides, such as explaining how job_id is used in similarity calculations or constraints on per_page values. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Find jobs similar to a given job listing') and the method ('using AI vector similarity'), distinguishing it from sibling tools like get_job (retrieve single job) and search_jobs (general search). It explicitly mentions the use case ('more like this' discovery), making the purpose highly specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Great for "more like this" discovery'), implying it's for similarity-based recommendations rather than direct retrieval or general search. However, it does not explicitly state when not to use it or name alternatives like search_jobs for different query types, which would be needed for a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.