Server Quality Checklist

Tool Scores

• verifyA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds a key behavioral trait beyond the readOnlyHint annotation: 'verify failures are data, not isError.' This tells the agent that failures are returned as data rather than errors, which is crucial for correct handling. It also notes the return of a structured report, providing useful context for tool invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two sentences: the first clearly states the purpose, and the second provides a crucial behavioral note. Every word earns its place, with no unnecessary information or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, annotations present), the description is sufficiently complete. It covers the tool's purpose, parameter, and a key behavioral aspect. However, it could be more complete by explicitly stating that it is a read-only verification tool and how it relates to sibling tools, but the current level is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a description for the 'paths' parameter: 'Code paths the agent intends to edit.' The tool description does not add any additional meaning or clarification beyond what is already in the schema, so it meets the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Verify the TNLs impacted by a set of code paths.' It specifies the verb (verify) and resource (TNLs impacted by code paths), and mentions the return of a structured report. However, it does not differentiate from siblings like 'get_impacted_tnls' which also deals with TNLs, slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool over alternatives such as 'get_impacted_tnls' or 'propose_tnl_diff'. It lacks explicit when-to-use or when-not-to-use conditions, leaving the agent to infer usage context from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_impacted_tnlsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, and the description adds the specific behavior that repo-wide units are always included. No contradictions. Could mention limits or error handling but fine for a simple retrieval.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy, front-loaded with the action verb 'Return'. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 param, no output schema), the description covers the essential behavior and scope. Minor gap: no mention of return format or pagination, but acceptable for a straightforward list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with clear parameter description. The tool description reinforces that paths are for overlap but adds no new semantic information beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool returns TNL units based on overlapping code paths, and always includes repo-wide units. This distinguishes it from siblings like retrieve_tnl or trace.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives like retrieve_tnl or trace. The description implies usage for determining impacted TNLs before edits but lacks when-not-to-use or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• propose_tnl_diffA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate it is not read-only, not destructive, not idempotent. The description adds that it returns a diff_id for later approval, but does not disclose side effects or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the purpose, and includes the key return value and next step. No extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description usefully mentions the returned diff_id. However, it could clarify that changes are staged but not applied immediately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters (intent, changes) are fully described in the input schema. The tool description adds no additional parameter-level information beyond the schema's descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Stage a proposed TNL change') and resource ('batch of creates and updates'), and distinguishes it from sibling tools like approve_tnl_diff by noting the returned diff_id.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used before approve_tnl_diff for human review, but does not explicitly state when not to use it or provide alternatives like get_impacted_tnls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retrieve_tnlA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond annotations by explaining that missing ids result in a 'notFound' list. Annotations already indicate readOnlyHint=true, and the description complements this by detailing the behavior for nonexistent ids.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the primary action. No extraneous text; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and no output schema, the description covers the key behavior: returning verbatim content and listing missing ids. It provides sufficient context for an agent to understand the response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the description 'TNL unit ids to fetch.' The description does not add further detail about the parameter format or constraints beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Return the verbatim content of TNL units identified by id,' specifying the action and resource. It distinguishes from sibling tools like approve_tnl_diff or propose_tnl_diff, which are mutation-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For instance, no mention of when to use get_impacted_tnls instead. The description only states functionality without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• approve_tnl_diffA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructive and idempotent; description explicitly lists file writes, sidecar regeneration, and staging removal—adding concrete behavioral specifics beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single 15-word sentence, no redundancy, every word carries meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given single param, no output schema, and rich annotations, description sufficiently covers actions and prerequisite. Minor omission: no mention of return value or error cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers diff_id with description, but description adds valuable origin context ('returned by propose_tnl_diff'), helping agent understand where the parameter comes from.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'Apply' and resource 'staged proposal', enumerating exact actions (write, regenerate sidecar, remove staging record). Clearly distinguishes from sibling tool propose_tnl_diff which creates the proposal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies prerequisite of having a staged proposal from propose_tnl_diff, but does not explicitly state when not to use or alternatives. Sibling context helps but could be more direct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• traceA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (which indicate the tool is not read-only, not destructive, and not idempotent), the description adds that the server overwrites any caller-supplied timestamp. This is a key behavioral detail for the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core functionality and wastes no words. It is maximally concise while still being informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the core behavior and parameter usage, it omits information about the return value when reading events. For a tool with no output schema, this is a gap: the agent doesn't know what the read operation returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, the description still adds value by explaining the conditional semantics of the `event` parameter: its presence triggers recording, absence triggers reading. The description clarifies the dual role beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's dual purpose of recording or reading session events related to TNL usage. It uses specific verbs ('record' and 'read') and a specific resource ('session events'), and it distinguishes itself from sibling tools like approve_tnl_diff or retrieve_tnl.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use each mode: pass `event` to record, omit it to read. This tells the agent exactly how to invoke the tool for the desired action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.