Server Quality Checklist

Tool Scores

• generate_signed_requestB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses two behavioral traits: automatic handling of query_hash branching and idempotency headers. However, it does not mention that the generated code is a safe, read-only operation or what side effects (e.g., actual API calls) might occur. The disclosure is partial but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, and immediately followed by a key behavioral detail. Every sentence is meaningful and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters and no output schema, the description is too sparse. It does not explain what the tool returns (the generated code snippet), nor does it mention prerequisites (e.g., possession of API keys). The agent lacks critical context for invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions. The description adds no extra parameter-level meaning beyond the schema; for example, it does not clarify how 'query_hash' or 'idempotency headers' relate to specific parameters. Baseline 3 is appropriate given the schema already does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'generates' and specifies the resource as 'complete signed code for calling specific endpoints', listing four languages. It clearly distinguishes from sibling tools like 'sign_request' (which signs an existing request) and 'verify_signature' (which verifies).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'sign_request' or 'signing_guide'. The description lacks explicit context for when-not-to-use or prerequisites, leaving the agent without decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sign_requestB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that secret_key is not transmitted (local only), which is a key safety trait. However, it does not mention whether the tool makes any network calls, what happens on invalid keys or missing inputs, or any error behavior. The positive safety statement is useful but leaves several behavioral aspects undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long and front-loads the main purpose (local JWT computation for debugging). It is compact, uses simple language, and conveys key points without redundancy. A minor improvement could be structuring the output components as a list, but overall it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 8 parameters, nested objects, no output schema, and no annotations, the description provides core purpose and safety but lacks a complete usage story. It does not explain how to construct inputs (e.g., which operationId to use, how to fill pathParams), nor does it detail the output format beyond naming three components. The description partially compensates but leaves gaps for a full understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for 5 out of 8 parameters (63% coverage), including hints for 'body', 'server', 'accessKey', 'secretKey', and 'pathParams'. The description adds overall purpose and a safety note about secret key locality but does not add detailed semantics for individual parameters beyond what the schema states. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool computes a JWT locally for a single request using real keys (debugging purpose). It specifies the result includes JWT, query_hash, and a ready-to-use curl command. However, it does not explicitly differentiate from the sibling tool 'generate_signed_request' or other alternatives, so purpose is clear but not fully positioned among siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description marks the tool for debugging ('디버깅용') and emphasizes that the secret key is used only locally, implying it is safe for testing. However, it does not explicitly state when to use this tool versus alternatives like 'generate_signed_request' or 'verify_signature', nor does it mention prerequisites or exclusions. Usage context is implied but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_signatureB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description partially fulfills the behavioral transparency burden. It discloses that verification is local ('로컬') and can diagnose query_hash mismatches. However, it does not mention idempotency, side effects (e.g., no mutation), required permissions, or response format. The description adds some value but is not fully comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence that packs purpose, method, and diagnostic scope without redundancy. It is front-loaded with the main action ('로컬 검증') and efficiently adds key constraints ('같은 순서로'). It could improve by breaking into two sentences for readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and no annotations, the description provides moderate coverage. It explains the diagnostic purpose and links two parameters to specific use cases, but omits mention of the 'method' parameter's role in verification logic, the 'token' prefix handling, and any post-verification output structure. Additional context on return values or errors would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 80%, so the baseline is 3. The description adds context by linking 'query' (GET verification) and 'rawBody' (POST hash comparison) to use cases, which goes beyond the raw schema descriptions. However, the description does not explain the 'method' parameter's role in hash computation or the 'token' parameter's format tolerance beyond what the schema notes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('verify', 'diagnose') and resources ('JWT', 'query_hash'), clearly stating it validates a JWT locally and identifies 401 causes, including query_hash mismatches. It distinguishes from sibling tools like 'sign_request' and 'generate_signed_request' which focus on creation, not verification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for debugging 401 errors and checking hash consistency, but does not explicitly state when to use this tool versus alternatives like 'signing_guide' for learning about signing, or 'list_error_codes' for error interpretation. No direct exclusions or prerequisites (e.g., requiring the secret key) are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_endpointsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It describes the output inclusions (scope, idempotency, rate limit) which is good, but does not disclose any behavioral traits such as caching, pagination, authentication requirements, or whether it is a safe read operation. Adequate but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, no wasted words. Front-loaded with the key verb 'list' and resource. Every sentence adds unique value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only one simple boolean parameter and no output schema, the description is complete enough for a listing endpoint. It names the key output fields (scope, idempotency, rate limit bucket) which helps the agent understand what to expect. No missing critical information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal extra meaning beyond the schema's description of the boolean parameter, only restating that default is true and that all endpoints are currently /open/*. No additional clarity on parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it lists Open API endpoints and provides scope, idempotency, and rate limit info. It clearly specifies the resource ('Open API 엔드포인트 목록') and what each item includes, but does not explicitly distinguish from siblings like 'get_endpoint' or 'list_error_codes'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is given, nor alternatives mentioned. The context of 'currently all /open/*' is helpful but does not guide the agent on when to prefer this over get_endpoint or other list tools. Implied usage for exploring endpoints, but lacks exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_endpointA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool returns details on authorization and idempotency (권한·멱등성), but does not clarify if the operation is read-only, how it handles missing operationId (though required), or any rate limits. The exposure of idempotency info hints at safety but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence listing clearly what details are returned. It is front-loaded with the core purpose. Could be considered slightly dense due to the list of attributes, but overall efficient. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one required parameter with full schema coverage, no output schema, and the description lists all major return categories, it provides sufficient context for an agent to invoke it correctly. The lone missing piece is a mention of the return format (e.g., JSON), but the example-driven description compensates well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds the example value 'buyAsset, getPrices, getPriceHistory' for the operationId parameter, which is helpful but does not go beyond what the schema already provides (a string with description). No additional semantics beyond the parameter name and example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states '단일 엔드포인트 상세' (single endpoint details) and lists the specific attributes returned: 설명, 파라미터, 요청 본문 스키마와 예제, 응답 상태, 권한·멱등성. This clearly distinguishes it from sibling tool 'list_endpoints' which likely returns a list of endpoints rather than details for one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when needing details of a specific endpoint by operationId. However, it does not explicitly contrast with alternatives like 'list_endpoints' (which lists all endpoints) or mention when not to use this tool (e.g., if you only need error codes, use 'list_error_codes' instead). The usage context is clear but lacks explicit exclusions or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• signing_guideA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully communicate behavioral traits. It describes the tool as a 'procedure' or guide, implying it is non-destructive and informational. It also adds context about debugging common signing errors. However, it does not explicitly state that the tool does not make API calls or have side effects, leaving minor ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences in Korean. The first sentence states the core purpose, and the second provides a critical usage hint (common failure point). No extraneous text; every sentence earns its place. It is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description adequately covers the tool's nature as a reference guide. It names the signing method (JWT + query_hash) and a specific error scenario. However, it does not describe the output format (e.g., return type or content structure), which could aid an agent in processing the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters with 100% schema coverage, so the baseline is 4. The description does not need to explain parameters. It adds no parameter-specific information, but none is required. The description's focus on the signing guide content is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as a procedure for request signing (JWT + query_hash). It also adds specific value by mentioning a common pitfall (401 errors when copying the Upbit example), which distinguishes it from sibling tools like sign_request that actually perform signing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly guides usage by highlighting the tool as a reference for developers stuck on 401 errors from copying examples. However, it does not explicitly state when not to use it (e.g., when actual signing is needed) or mention alternatives like generate_signed_request. The context provides some inference, but lacks direct exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_error_codesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully bears the burden of behavioral disclosure. It explicitly discloses specific error code pitfalls (e.g., insufficient balance, authentication failure) and states the inclusion of a retry judgment table, which informs the agent of the tool's scope and limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise and front-loaded, but it uses a non-standard format (parenthetical examples, semicolons). The key information is present, but the structure could be clearer for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no output schema, and no annotations, the description is complete. It explains the tool's purpose, lists concrete examples of what it reveals (specific error codes, pitfalls), and mentions the retry table. There are no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema description coverage is 100% (empty schema). The description adds no parameter information because none is needed. This is a baseline 4, upgraded to 5 because the description's content about error codes and retry table is exactly what the agent needs to know for this parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: it lists Open API error codes and common pitfalls (e.g., insufficient balance returns 400 P0001, authentication failure returns error=null 401, 503 fail-closed). This distinguishes it from siblings like list_endpoints or signing_guide.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions it includes a retry judgment table by status code, implying it should be used for understanding error handling and retry logic. However, no explicit guidance is given on when to use this tool versus siblings like get_endpoint or signing_guide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.