Server Quality Checklist

Tool Scores

• cancel_scheduled_sendB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the required permission but does not explain what happens after cancellation, whether it is reversible, or any other behavioral traits like rate limits or side effects. The output is not described despite an output schema existing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with one sentence and a permission note. No unnecessary words, but it sacrifices completeness. It is front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is simple with one parameter and an output schema exists, the description is incomplete. It fails to explain the return value, how to identify the scheduled send, or any post-cancellation effects. For a tool with no annotations, this is inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning to the parameter 'invoice'. However, the description does not mention the parameter at all, leaving the agent without any guidance on what the invoice refers to or how to obtain its value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Cancel') and the resource ('a previously scheduled send'), and it distinguishes from sibling tools like schedule_send and send_invoice through the context of cancelling a scheduled operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a permission requirement ('Requires invoices:send'), which is a usage guideline, but does not explicitly state when to use this tool versus alternatives or when not to use it. The context of cancelling a scheduled send is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reminder_copyB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only says 'Show'. It does not disclose that the tool is read-only, whether authentication is needed, or what happens if no copy exists. The description carries the full burden but is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no fluff. It could be slightly expanded to add context without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, existence of an output schema, and sibling tools, the description is minimally adequate. However, it could better explain the purpose (e.g., preview vs. sending).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description cannot add meaning beyond the schema. Baseline 4 applies per guidelines.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool shows the user's custom reminder email copy and schedule, using a specific verb and resource. It distinguishes itself from siblings like 'send_reminder' which actually sends the reminder.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'send_reminder' or 'schedule_send'. Agents must infer from the description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_clientA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses partial update behavior and the required permission, but does not mention other behaviors such as idempotency, side effects, rate limits, or what happens when required fields are omitted. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no fluff. Every sentence adds value: purpose and partial update rule. Efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, 1 required, and no enums, the description misses contextual details like return format (though output schema exists), usage scenarios, and parameter interdependencies. Adequate for a simple update but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description should compensate. However, it only states that changes are field-specific and lists no parameter semantics (e.g., default_rate meaning). The schema provides titles but no descriptions, leaving agents to guess parameter meanings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'update' and resource 'client details', and distinguishes from sibling tools like delete_client. It also specifies the partial update behavior ('Only the fields you pass change'), leaving no ambiguity about the tool's purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says when to use (update client details) and that it requires clients:write permission, but does not explicitly state when not to use or mention alternative tools. The context of sibling tools provides implicit differentiation, but explicit guidance is lacking.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_sendA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. Mentions 'Client-facing' and permission requirement. However, lacks details on side effects (e.g., what happens to draft state, whether scheduling can be modified) and does not address edge cases like past scheduled_at. Provides basic but not comprehensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences covering purpose, param format, usage condition, and permission. Front-loaded and efficient. Minor improvement could be grouping permissions with usage condition for better structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists (so return values not needed) and sibling tools provide context, the description adequately covers prerequisites (draft invoice, explicit confirmation) and distinguishes from related actions. Still, could explicitly mention that invoice must be in draft state and not already scheduled.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% coverage; description partially compensates by explaining `scheduled_at` format with an example. However, `invoice` parameter is not described (likely an ID or reference). Baseline 3 is appropriate, but could be improved by clarifying `invoice`.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'Schedule' and resource 'DRAFT invoice to be emailed at a future time'. Distinguishes from siblings like 'send_invoice' (immediate) and 'cancel_scheduled_send' (cancellation), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call ('only when user has explicitly confirmed') and includes a permission requirement ('Requires invoices:send'). Could further clarify when not to use (e.g., if invoice is already sent), but overall guidance is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_hoursA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It discloses partial update behavior ('Only the fields you pass are changed') and the invoice lock condition. It does not cover rate limits or other side effects, but the provided info is sufficient for safe operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three focused sentences: first states purpose, second adds behavioral nuance, third lists requirement. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters (1 required) and an existing output schema, the description covers main behaviors and constraints but lacks detailed parameter explanations. It adequately informs about the partial update and invoice lock, but more details on editable fields would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only mentions that only passed fields are changed, without explaining individual parameters like date, rate, hours, etc. The description fails to add meaning beyond the schema for each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Edit an existing time entry', specifying the verb (edit) and resource (existing time entry). It distinguishes from sibling tools like 'log_hours' which creates new entries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises confirming with the user before editing entries on sent/paid invoices due to a server block, providing clear contextual guidance. It also lists the required permission 'timesheet:write', but does not explicitly state when not to use the tool or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• log_hoursA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully convey behavior. It mentions required permission ('timesheet:write key'), which is important. But does not disclose idempotency, whether it overwrites existing entries, or any side effects beyond basic creation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: first states purpose, second explains key parameter details, third states permission requirement. No redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters and 4 required, description covers basic usage. However, lacks details on output (though output schema exists), error conditions, or concurrency behavior. Slightly incomplete for a tool with this complexity, but acceptable with output schema present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description adds value by detailing 'rate' behavior and 'date' format. However, it does not explain 'client', 'hours', 'description', or 'project' beyond their names, leaving some parameters semantically underdocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the action (log), the resource (worked hours for a client), and the system (Peil's timesheet). Distinguishes from sibling 'edit_hours' by using 'log' instead of 'edit', implying creation of new entry.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on optional 'rate' parameter (omit to use default) and date format (ISO YYYY-MM-DD). However, does not explicitly contrast with sibling 'edit_hours' or specify when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_reminderA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses irreversibility and client-facing nature, which are critical behavioral traits. It also mentions permission requirement. No annotations provided, so description carries full burden and does so adequately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, highly concise. The first sentence captures the action and key traits; the second adds usage guidance. No extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and an output schema. The description covers purpose, usage condition, and consequences. It does not mention output format, but that is likely in the output schema. Slightly lacking in parameter details, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (invoice) with 0% schema description coverage. The description adds no specific guidance on the parameter's format, source, or validation. The context implies it's an invoice identifier, but the description should compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact action: email a payment reminder for sent/overdue invoices. It clearly distinguishes from siblings like send_invoice or schedule_send by specifying the reminder context and noting irreversibility.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (only after user confirmation) and a prerequisite (requires invoices:send permission). Does not explicitly mention when not to use or alternatives, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_clientA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses destructive behavior and conditions for failure. No annotations exist, so description carries full burden. Could mention if deletion is irreversible or affects related data, but the blocking conditions mitigate risk.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: three short sentences, front-loaded with purpose. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simplicity (1 param, no nested objects) and presence of output schema, the description adequately covers purpose, constraints, and authorization. Lacks detail on parameter but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'client' is not described beyond the schema's title. With 0% schema description coverage, the description should compensate but does not add any meaning about the parameter (e.g., what value to provide).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states clearly 'Delete a client.' with a specific verb and resource. It distinguishes from siblings like 'update_client' by implying destruction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly warns about destructive nature ('confirm first'), conditions that block deletion (409 if has projects/invoices), and required permission ('clients:write'). This provides clear when-to-use and when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_invoiceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, so description carries full burden. States irreversible and client-facing nature, plus permission requirement. Provides sufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise, front-loaded with key info, every sentence adds value. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple send tool with output schema, description covers purpose, usage, parameter, and behavior. Missing details on email validation or confirmation flow, but adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter 'invoice' has 0% schema coverage, but description adds meaning: numeric id or invoice number. Clarifies input beyond schema type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Send', resource 'invoice', and medium 'BY EMAIL'. Distinguishes from sibling tools like send_reminder and schedule_send.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call (only after explicit user confirmation) and permission requirement. Could mention alternatives but not needed given clarity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.