Server Quality Checklist

Tool Scores

• update_incidentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for behavioral disclosure. It clearly states that this is a mutating operation (update), explains the effect of each parameter, and warns that resolving/closing may be rejected by the backend. It does not mention idempotency, error handling for missing incidents, or authentication requirements, but the provided details are sufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. It starts with a one-sentence summary of the action, then a one-sentence usage context, then a constraint, then a clear Args/Returns section. Every sentence adds unique information; there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all parameters, provides a return format, and includes a key behavioral caveat (rejection on missing resolution fields). It is almost complete for a mutation tool with no annotations, but it does not specify what happens if the incident number does not exist or whether the operation is idempotent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must fully explain each parameter. It does so: number is given an example ('INC0010001'), work_notes is described as 'append to the (internal) work notes journal', and state is documented with both numeric codes and names, plus a note about additional fields required by ServiceNow. This adds significant semantic value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear verb ('update'), resource ('existing incident'), and method ('by number'). It explicitly lists the two possible actions (append work notes and/or change state), which distinguishes it from siblings create_incident and query_incidents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises when to use the tool ('record progress on a ticket or move it through its lifecycle') and states a critical constraint ('at least one of work_notes or state must be provided'). It also warns about potential rejection when resolving/closing if resolution fields are missing. However, it does not explicitly contrast with alternatives like create_incident or query_incidents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_incidentA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that the tool creates a record and returns specific fields. It could add potential side effects or permissions needed, but is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured, front-loaded with clear purpose and usage. Each sentence adds value, no fluff. Parameters and return value explained efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has 3 parameters and output schema exists. Description covers parameter semantics, return format, and usage context. No gaps given complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It does so by explaining each parameter's purpose and format (e.g., urgency values, description use). Adds meaning beyond schema (e.g., 'shown in lists'). Could note default for short_description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a ServiceNow incident and returns its number and sys_id. It distinguishes itself from siblings by specifying creation, not querying or updating.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool: 'when a user reports a problem that should be tracked as a ticket.' No exclusion of siblings but clear context for usage, and siblings are named for differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_incidentsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral details: search semantics (exact number matching vs. keyword search with substring fallback), state filter format (numeric or name), limit behavior with default, and return format. It also explains the meaning of an empty list. This is comprehensive and goes beyond a simple 'lists incidents.'

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized into clear sections: a one-line summary, usage guidance, parameter details, and return format. Every sentence serves a purpose, from explaining search fallback to specifying defaults. It is appropriately sized given the behavioral complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a read-only query tool: it defines input semantics, output structure, and edge cases (empty list, empty query). The presence of an output schema is complemented by a human-readable description of the return fields. Given the sibling tools, it clearly fits into the workflow as the search/read operation. No gaps are apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain parameters. The 'Args' section does exactly that: query_text, state, and limit are each described with accepted formats, defaults, and special behavior (e.g., prefix matching for incident numbers, state code ranges). This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List ServiceNow incidents, newest first, optionally filtered by text and/or state.' This uses a specific verb, names the resource (ServiceNow incidents), and distinguishes it from sibling tools like create_incident and update_incident by focusing on listing/querying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides use cases: 'Use this to find existing tickets before creating a new one, to check on a ticket by number, or to answer "what incidents are open about X?".' This gives clear when-to-use guidance. It implies when-not-to-use (e.g., before creating) but does not explicitly name alternatives or exclude other tools, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.