Server Quality Checklist

Tool Scores

• filter_jobs_by_skillsC
  2.4/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations and a minimal description, the behavioral impact is completely undisclosed. The tool could be a read-only filter or perform hidden side effects; the description gives no information about permissions, errors, return format, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with no redundancy or unnecessary words. It is extremely concise and well-structured for a tool of this simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks essential context for an agent to use the tool correctly: it does not specify the output format, error handling, pagination details, or how it differs from 'search_jobs'. Without such information, the tool is not fully usable in a real workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only the 'skills' parameter is partially explained by the description ('list of required skills'), but 'limit' is not mentioned at all. The description does not clarify what the limit controls (e.g., max results, pagination), so the meaning of this parameter is left entirely to inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (filter), the resource (jobs), and the criterion (by list of required skills). It is specific enough to understand the basic purpose, but it does not explicitly differentiate from the sibling tool 'search_jobs', so it loses one point for lacking that distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the alternative 'search_jobs'. It implies a filtering use case, but does not give explicit conditions or preferences, leaving the agent to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_jobsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'Search' and does not mention pagination, filter interactions, or any operational behavior beyond what the name itself already implies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler, which is concise. However, the brevity crosses into under-specification because it omits nearly all operational detail needed to use the tool effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 11 parameters, no annotations, and zero parameter descriptions, yet the description only identifies the domain. The output schema covers return values, but usage criteria, parameter semantics, and sibling differentiation are all unaddressed, making the tool difficult to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description names none of the 11 parameters such as query, remote, limit, or page. The agent receives no semantic help beyond parameter titles and defaults, which is insufficient for such a parameter-heavy tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search') and a clear resource ('YC startup jobs on WorkAtAStartup'), so an agent knows what the tool does. It does not explicitly contrast it with filter_jobs_by_skills, whose purpose could overlap, so it does not fully achieve sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance about when to use this tool versus siblings like filter_jobs_by_skills or get_job_details. It states neither preconditions, exclusions, nor conditions that would help route an agent to the correct tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_company_detailsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It only says 'Get' which implies read-only, but does not disclose error behavior, authentication requirements, rate limits, or what 'full details' includes. Minimal behavioral context is given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, front-loaded, with no filler. It efficiently states the purpose and parameter context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (get by ID) and the presence of an output schema (though not shown), the description is minimal. It lacks any edge-case or error behavior, but for a basic retrieval it may be sufficient. However, it doesn't mention return format or any other context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description should compensate. It does say 'by ID' which aligns with the parameter, but it does not add any format constraints, requiredness, or examples beyond what the schema already shows. For a single self-explanatory parameter, this is adequate but not enriching.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get') and resource ('full company details') keyed by ID. It is clearly distinct from the siblings (jobs-related), so an agent can immediately identify its purpose without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives. While the sibling tools are about jobs)Skip the clear differentiation, there is no explicit statement of when this tool is appropriate or what it is not for.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_job_detailsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the burden. 'Get' implies a read operation, which is safe, but the description doesn't disclose any return format, error behavior, or access requirements. It's a simple read, so the lack of detail is not critical, but it could be more transparent about the output structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, extremely concise and direct. It provides the necessary information without any fluff. The purpose is front-loaded with the verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema specifics), the description is sufficiently complete. The agent likely knows what job details are (e.g., title, status, maybe timestamps) from the output schema (not provided to me but present in the actual definition). Since there is an output schema, the description doesn't need to explain return values. The only gap is the lack of differentiation from sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning the schema doesn't explain what 'job_id' is beyond its type. The description adds the meaning that the tool retrieves details based on this ID, which is essential. Since there is only one parameter and a clear description, the agent understands what to provide. However, it doesn't go beyond that basic meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get') and resource ('job details') with a clear parameter (by ID). It clearly identifies what the tool does. However, it does not distinguish this from sibling tools like 'get_company_details' or 'search_jobs', though the focus on 'job details' is somewhat distinctive.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description doesn't mention that this is for fetching a single job's details, while 'search_jobs' is for querying, or 'filter_jobs_by_skills' for filtering. No exclusions, but also no clear context to differentiate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.