Server Quality Checklist

Tool Scores

• nhcx_process_caseB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It mentions that the usecase is derived automatically, but does not disclose side effects, irreversibility, authentication requirements, or return format. For a tool that can approve/reject cases, this is a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at two sentences, front-loading the main purpose and action list. The second sentence efficiently lists required parameters. No fluff or redundant information, though the structure could be slightly improved with a clearer separation of purpose and usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool performs case mutations and there is no output schema or annotations, the description is incomplete. It lacks information about expected responses, error handling, pre-conditions (e.g., login), or post-condition effects. The schema covers parameters but not behavior, and the short description does not compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal semantic value beyond the schema, simply restating that certain parameters are required. The schema already provides clear descriptions for each parameter, including the critical note about never guessing correlationId.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool acts as a payer and performs an action on a case, listing the specific actions. This distinguishes it from sibling tools like nhcx_login or nhcx_workflow, which serve different purposes. The verb+resource is specific enough, though it could be crisper.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for case processing actions but does not explicitly say when to use this tool versus alternatives. It mentions required parameters, which helps, but there are no exclusion criteria or explicit mention of when to prefer sibling tools. The guidance is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nhcx_get_user_roleA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It indicates that the tool 'gets' role and allowed actions, but does not disclose side effects (e.g., read-only), authentication requirements, rate limits, or error behavior. The 'Always call' note is workflow guidance rather than behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: one states the purpose, the other gives a clear directive. Every word earns its place, and the critical usage note is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with no output schema, the description explains what is returned (role and allowed actions) and when to call it (before processing). It lacks mention of prerequisites like login, but the sibling list includes nhcx_login, and the core information is sufficient for most use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for both parameters, so the description adds little beyond the schema. The phrase 'for a case' maps to caseId, but no new meaning is added for payerId or the parameters in general. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and clearly identifies the resource ('current payer role for a case and the allowed actions for that role'). This distinguishes it from sibling tools like nhcx_login (auth) and nhcx_process_case (processing), making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Always call this before processing a case' provides clear context on when to use the tool relative to processing a case. However, it does not mention when not to use it or explicitly name alternatives, so it falls slightly short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nhcx_loginA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the authentication action, token caching, and a 15-minute timeout, which is valuable. However, it does not explain what happens on failure, whether an existing token is overwritten, or any potential side effects beyond caching.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, front-loaded with the core purpose and followed by the use case. Every phrase adds value, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter tool with no output schema, the description effectively covers purpose, usage guidance, and a key behavioral detail (token lifetime). It could mention the return value or prerequisites, but the note that other tools auto-authenticate makes the overall context sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides complete information. Per the rubric, a baseline of 4 is appropriate since there is nothing for the description to add about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Authenticate with the ABDM gateway V3 and cache a session token (15 min).' It uses a specific verb and resource, and distinguishes itself from siblings by noting 'all other tools auto-authenticate', making its purpose unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Useful as a connectivity check' and 'all other tools auto-authenticate', providing both a recommendation for when to use it and an implicit exclusion for normal operations. This is clear guidance versus alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nhcx_validate_configA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It clearly states the actions performed (validating environment variables and testing API connectivity) without implying any hidden side effects. The behavior is unambiguous, though it does not explicitly state whether the operation is read-only or what happens on failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the tool's action. It contains no redundant information and efficiently conveys both purpose and usage timing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description adequately covers its purpose and usage. However, it does not specify what output or exit status the user should expect after validation, which would enhance completeness but is not critical for a straightforward validation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and per the guidelines, no parameter description is needed. The description's mention of 'environment variables' and 'API connectivity' indirectly clarifies what is being validated, but since there are no parameters, a baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with specific verbs and resources: 'Validate all environment variables and test connectivity to NHCX APIs.' This distinctly separates it from sibling tools like nhcx_login and nhcx_process_case, and 'Run this first' signals its role as a precursor step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage timing with 'Run this first to confirm your setup is correct.' This gives clear context for when to use the tool, though it does not mention specific alternatives or exclusion cases, which are less necessary given its distinct validation role.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nhcx_workflowA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Show' implies a read-only operation, and the description details what content is returned (steps, usecase, roles, actions). It does not mention authentication or potential side effects, but for a read-only workflow display, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the verb and resource, densely listing the included details. There is no filler or redundancy, making it exceptionally concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description fully specifies what the tool shows (steps, usecase per step, roles, allowed actions). This is sufficient for an agent to understand the tool's behavior fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters and an empty schema, the schema coverage is 100% (vacuously). Baseline for 0 params is 4. The description need not explain parameters, and it doesn't.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Show') and resource ('full payer workflow'), and enumerates the included content (steps, usecase per step, roles, allowed actions). This clearly distinguishes it from sibling tools like nhcx_process_case or nhcx_login.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly conveys that this tool is for viewing workflow information, contrasting with the action-oriented siblings. However, it does not explicitly provide when-to-use or when-not-to-use scenarios or name alternatives, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.