Server Quality Checklist

Tool Scores

• estimate_annual_costA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include readOnlyHint=true and openWorldHint=false, covering the safety profile. The description adds meaningful behavioral context by specifying that the output covers both monthly and annual costs for both a person and the AI employee, within one market. This goes beyond a simple read-only declaration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences only. The first sentence front-loads the core function, and the second states when to use it. Every word earns its place; no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description compensates by indicating the tool returns monthly and annual costs for both person and AI employee. The inputs are fully documented in the schema, and the purpose is clear. It does not detail currency or formatting, but for a prediction/estimation tool the description is sufficient for selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all three parameters well-described. The description reinforces the meaning of unitsPerMonth ('monthly volume') and market ('one market') but does not add new parameter-level details beyond the schema. This is baseline-appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Turn') and clearly states the resource and scope: 'a monthly volume of a task into monthly and annual cost, for a person and for the AI employee, in one market.' This distinguishes it from siblings like get_task_cost, which likely provides only a single cost figure, and list_work_tasks, which lists tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The second sentence provides explicit usage context: 'Use this when someone asks what a workload costs them per year, or what they would save.' It does not explicitly mention when not to use it or name alternatives, but the guidance is clear enough for the intended use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cost_of_work_indexA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, covering the safety profile. The description adds that the return format is JSON and that it includes caveats, but does not elaborate on pagination, size, or other behavioral details. Since the annotation covers the primary behavioral concern (read-only), the description adds minimal extra context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the tool's purpose and followed by usage guidance. Every word earns its place, with no tautology or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (0 required parameters) with a read-only annotation and no output schema. The description adequately communicates the scope of the returned dataset ('every task, every market, every source') and the presence of caveats. While the exact structure of the JSON is not described, it is not essential for a simple retrieval tool. The only minor gap is the vague 'caveats' reference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter `market` is fully documented in the input schema, including its optional nature and valid values (`us`, `gr`, `ua`). The description does not add additional meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's output as the complete Cost of Work Index as JSON, specifying 'every task, every market, every source'. It differentiates from sibling get_task_cost by noting when to prefer a single figure. The title 'Get the whole Index as JSON' reinforces the specific verb and resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the use case: 'when you need the dataset itself rather than one answer'. It names an alternative tool, `get_task_cost`, for a single figure, providing a clear when-to-use and when-not-to-use scenario. This meets the threshold for explicit usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_task_costA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds meaningful behavioral context: the exact output components (human/AI cost, ratio, source) and the important market naming exception. This goes beyond the structured fields without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, both information-dense: the first defines the output, the second explains market values and a critical caveat. No wasted words or redundancy with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple read-only nature, two parameters, and no output schema, the description fully covers the return values and the market ambiguity. It leaves no significant gaps for a caller to understand what they will receive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds country names for market codes and explicitly warns that 'uk' means Ukraine, enhancing the schema's terse alias list. This extra clarity justifies a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving the cost of one unit of a task in a specific market, and enumerates the returned figures (human cost, AI cost, ratio, source). This distinguishes it from sibling tools like estimate_annual_cost, which implies a broader annual scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for single task-market unit costs but does not explicitly contrast with sibling tools like estimate_annual_cost or get_cost_of_work_index. It offers useful context about market codes and the 'uk' caveat, but no when-to-use/when-not-to-use guidance relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sourcesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint=true already indicates a safe read operation. The description adds context about data provenance and contents but doesn't disclose additional behavioral details like pagination or ordering, so it remains at a baseline level.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action, and every sentence adds value—no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple list operation, and the description covers what data is returned and when to use it. No output schema exists, but the description sufficiently conveys the tool's purpose and result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the schema already covers everything. The baseline for 0 params is 4, and the description adds no parameter-specific information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List the Index sources' and details the specific fields (publisher, release title, URL, reference period, and how the published figure became the number used here), making it distinct from sibling tools about tasks/costs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs 'Call this when you need to cite or verify a figure,' providing a clear use case. It doesn't mention alternatives, but the context is unambiguous given the distinct sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_work_tasksA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds useful behavioral context by stating the tool lists 'every' task and specifying the output fields. This goes beyond the annotation without contradicting it, though it does not disclose pagination or result format details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with a front-loaded verb 'List,' followed by the key output fields and a clear usage directive. Every sentence serves a purpose, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only listing tool, the description fully covers what it does, what it returns, and when to call it. The absence of an output schema is acceptable because the described fields are sufficient for the agent's needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is empty. The baseline for 0 params is 4, and the description does not need to add parameter semantics because there are none to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with the resource 'every task in the Cost of Work Index' and enumerates the returned fields (task id, unit, AI employee). This clearly distinguishes it from sibling tools like get_task_cost or estimate_annual_cost, which focus on cost calculations rather than listing tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call this first when you do not already know the task id you need,' providing a clear condition for use. It implies a sequencing among tools but does not explicitly name alternative tools or when not to use this tool, so it falls short of a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.