Server Quality Checklist

Tool Scores

• get_jobs_7_daysC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, but it only says 'Search and retrieve jobs.' It does not disclose that results are limited to the last 7 days, the 1–2 hour ingestion delay noted in the date_filter parameter, or how results are returned (pagination, format).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler, which is efficient. However, its brevity sacrifices critical context like the 7-day window, so it is not optimally front-loaded with the most distinguishing information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (33 parameters, no annotations, no output schema), the description is far too sparse. It omits the temporal scope, fails to disambiguate from neighboring tools, and does not mention the extensive filtering options or expected return value, leaving the schema to do all the heavy lifting.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with detailed descriptions for all 33 parameters, so the baseline is 3. The description adds a bit of grouping by naming a few searchable fields and mentioning 'text or html description', but it does not go beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a verb ('Search and retrieve') and resource ('jobs') and lists several searchable fields (title, description, location, remote, organization). However, it fails to mention the defining 'last 7 days' time scope, which is what distinguishes it from sibling get_jobs_24h, so it lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_jobs_24h or the ultra variants. There are no exclusions, prerequisites, or context about the intended use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ultra_3B
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'backfill,' which implies a read operation, but does not disclose rate limits, pagination behavior, response format, or any operational constraints. The parameter descriptions in the schema mention job credits and offset, but the tool-level description itself lacks this transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundancy, which earns points for conciseness. However, given the enormous parameter surface, it is under-specified and lacks structure, making it borderline between minimal and acceptable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 30 parameters, no annotations, and no output schema, the one-sentence description is inadequate for understanding the endpoint's full context. It does not explain what the endpoint actually returns, how it fits into the ultra family, or which parameters are most relevant for a backfill use case. The rich parameter descriptions help, but the overall tool description is far from complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage with descriptions for all 30 parameters, so the baseline is 3. The tool description adds no additional parameter-level semantics, but the schema descriptions are detailed enough to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('backfill') and a resource ('job application'), and it distinguishes this endpoint from sibling endpoints ('7d, 24h, or hourly endpoint'). This makes the general purpose clear, though 'backfill your job application' is somewhat ambiguous regarding whether it retrieves job postings or fills an application.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear usage directive: use this endpoint for backfill before moving on to the 7d, 24h, or hourly endpoints. This implies a temporal or sequencing relationship and names the alternatives, which provides when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jobs_24hB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of behavioral disclosure. It only mentions the 24-hour indexing freshness. It does not disclose pagination behavior, default limits, response format, or rate limits. This is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the action ('Search and retrieve'). No fluff. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 33 parameters and no output schema, the description is minimal but not inadequate. It gives a high-level overview and time window, with the detailed parameter schema filling in the rest. However, it doesn't describe the return structure or provide examples, leaving some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds a hint about optional text/html descriptions and lists search fields, but the parameter descriptions are already comprehensive. No additional parameter semantics in the main description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search and retrieve jobs on title, description, location, remote, and organization.' It also specifies the 24-hour indexing window, differentiating it from the 7-day sibling. However, it doesn't explicitly mention the breadth of available filters beyond the key fields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for jobs indexed in the last 24 hours, which is a usage context. However, it does not explicitly state when to prefer this over get_jobs_7_days or other siblings, nor does it provide exclusions or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ultraB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the full burden of explaining behavior. It only mentions deduplication when run daily, but omits key details such as response format, pagination behavior, rate limits, and the exact meaning of 'modified'. For a 31-parameter tool, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, front-loaded with the core functionality and followed by a usage recommendation. There is no filler or redundancy, and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 31 optional parameters, no output schema, and no annotations, the description is far too brief. It does not explain the rich filtering capabilities, response structure, or how this tool relates to ultra_2/ultra_3 siblings. This leaves an agent under-informed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The tool description adds no parameter-specific guidance, and many schema entries are merely 'Example value: ' with little value. However, the description is not obliged to duplicate schema content.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (search/retrieve), resource (jobs), and time scope (last 24h). However, it does not differentiate itself from siblings like get_jobs_24h or ultra_2, which likely have similar purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a specific usage pattern: run daily at the same time to get newly modified jobs without duplicates. This gives clear context for when to use the tool, but does not mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ultra_2A
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key requirement—'Requires Ultra or Mega subscription'—and the time window, but it does not describe read-only behavior, pagination defaults, response size, or other runtime characteristics. This is useful but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two short sentences, front-loaded with the core identity ('Firehose API') and each sentence adds value—functionality and usage/subscription requirement. No wasted words or repetition of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (31 parameters, no annotations, no output schema) and multiple similar siblings, the description provides only the essential time-window and subscription context. It lacks explanation of response format, default behavior, or differentiation from the ultra_3/ultra_4 variants, leaving the tool only partially complete for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 31 parameters. The description itself adds no parameter-specific meaning, which is acceptable under the baseline, though some schema descriptions contain only 'Example value:' placeholders that the description does not clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool is a 'Firehose API' that includes jobs discovered in the last hour, which clearly identifies its resource and time scope. It distinguishes from get_jobs_24h and get_jobs_7_days by emphasizing hourly freshness, but it does not clarify how it differs from sibling tools ultra, ultra_3, and ultra_4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Perfect for one or more hourly API calls', giving clear context for when to use it: for hourly incremental pulls. However, it does not name alternative tools or explicitly state when not to use it, so it falls short of full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ultra_4A
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It adds valuable behavioral context: the update schedule (once per day at 01:30 UTC) and the subscription requirement. It also implies a read-only operation by saying 'contains,' though it does not explicitly state side effects or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, followed by operational details. No filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description covers the essential context: what data is returned (IDs), the timeframe (yesterday), the refresh schedule, and access requirements. It does not specify the exact return format (e.g., array vs. object), but for a simple no-param tool, the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to add parameter meaning. The baseline for 0 params is 4, and the description does not detract from that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool contains IDs of jobs flagged as expired yesterday, specifying the timeframe and resource. It distinguishes from sibling tools by noting yesterday's expired jobs and the subscription requirement, though the verb 'contains' is less action-oriented than 'lists' or 'returns'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear prerequisite (Ultra or Mega subscription) and the update cadence (daily at 01:30 UTC), which helps with when to call. However, it does not explicitly contrast with sibling tools like get_jobs_24h or get_jobs_7_days, leaving it to the reader to infer the right use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.