Server Quality Checklist

Tool Scores

• nyx_list_servicesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must carry burden. It mentions that include_endpoints adds latency and that agent-key scope affects visibility. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: purpose then usage and return fields. No wasted words, well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description lists all return fields (slug, display name, etc.) and mentions endpoint inclusion condition. Covers parameter effects and usage context completely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage 100% means schema already describes parameters. Description adds latency note for include_endpoints but otherwise no additional semantics beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it enumerates downstream services with specific fields returned (slug, display name, etc.). Differentiates from siblings by mentioning tool discovery before nyx_proxy_request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this for tool discovery before issuing nyx_proxy_request' and notes agent-key scope determines visibility. Does not explicitly state when not to use, but context makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nyx_proxy_requestA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It discloses credential injection, rate limiting, audit logging, and supported service types. It does not detail the response format or error handling, but covers key behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no waste. Purpose is front-loaded, followed by capability examples and constraints. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should mention the response format (e.g., returns the downstream response). It also omits details like timeout limits or size constraints. Adequate for a proxy tool but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, providing baseline 3. The description adds meaning: body encoding rules, header restrictions, path base URL prepending, and service slug enumeration via another tool. This exceeds the schema's descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool forwards HTTP requests through NyxID to downstream services with credential injection. It lists supported service types (cloud APIs, internal endpoints, localhost) and distinguishes itself from sibling tools like nyx_list_services by referencing them for service enumeration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (accessing downstream services) and includes explicit guidance to not set Authorization headers. It also references nyx_list_services for available service slugs. However, it does not explicitly contrast with nyx_exchange_identity or nyx_request_approval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nyx_exchange_identityA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool performs token exchange, does not hold user credentials, and that the token carries user identity claims while remaining auditable. It doesn't cover rate limits or auth details, but is transparent about the core behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences only, with key information front-loaded. No wasted words, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the purpose, mechanics, and security implications, but could mention prerequisites like existing delegation rights. Nonetheless, it is reasonably complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions. The description adds context beyond the schema, explaining the overall delegation and auditing aspects, which enhances understanding of how parameters like subject_user and audience are used.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: exchanging the agent's NyxID identity for a delegated access token bound to a downstream OIDC service, citing RFC 8693. It uses a specific verb ('exchange') and resource, and distinguishes from sibling tools like nyx_list_services and nyx_proxy_request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the use case: letting the agent call APIs on behalf of a user without holding long-lived credentials. It provides clear context but does not explicitly mention when not to use or compare to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nyx_request_approvalA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses key behaviors: it blocks the agent, delivers via push notification, and times out. However, it does not describe the return value or how the agent receives the decision (e.g., approved/denied), which is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four efficient sentences front-load the purpose and mechanism, then provide usage guidance and options. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, behavior, and parameter constraints. However, it omits any mention of the tool's output or return value, which is needed since no output schema exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents parameters well. The description adds some context (e.g., timeout bounded by user max), but does not significantly enhance understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Request human approval before performing a sensitive action,' specifying the verb and resource. It differentiates from sibling tools like nyx_exchange_identity and nyx_proxy_request by focusing on human-in-the-loop approval for sensitive actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'for destructive operations, financial transactions, or any action where unattended automation is inappropriate.' Also indicates alternative approval methods via pre-configured grant rules, helping the agent decide contextually.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.