Server Quality Checklist

Tool Scores

• jwt_generateC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool generates a JWT but doesn't disclose behavioral traits such as whether it requires specific permissions, what the output format is (e.g., string), error handling, or security implications (e.g., key storage). The mention of algorithm types (HS* or RS*) adds minimal context but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and parameters without unnecessary words. Every part earns its place by specifying the tool's function and inputs clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of JWT generation with 4 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on output (e.g., JWT string format), error cases, algorithm specifics, and security considerations. For a cryptographic tool with nested objects in the schema, more context is needed for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists the four parameters (header, payload, algorithm, key) and specifies algorithm types (HS* or RS*), adding some meaning beyond the bare schema. However, it doesn't explain what each parameter represents (e.g., header fields like 'typ', payload claims), expected formats, or examples, leaving significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Generate' and the resource 'JWT', specifying the four input parameters (header, payload, algorithm, key). It distinguishes from siblings like jwt_analyze, jwt_bruteforce, and jwt_decode by focusing on creation rather than analysis or decryption. However, it doesn't explicitly contrast with siblings beyond the different action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like jwt_analyze or jwt_decode. It mentions algorithm types (HS* or RS*) but doesn't explain when to choose this over other JWT tools or what scenarios warrant JWT generation. There's no mention of prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jwt_analyzeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions analyzing for 'common vulnerabilities and issues,' which implies a read-only, diagnostic operation, but doesn't specify details like what vulnerabilities are checked, whether it modifies the token, potential rate limits, or output format. For a security tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence: 'Analyze a JWT for common vulnerabilities and issues.' It's front-loaded with the core purpose, has zero wasted words, and is appropriately sized for a simple tool. Every part of the sentence contributes directly to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (security analysis tool), lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like what specific vulnerabilities are analyzed, whether it's safe for production use, or what the output looks like. For a tool with potential security implications and no structured data to rely on, this leaves the agent under-informed about critical context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter (token) with 0% description coverage, so the schema provides no semantic information. The description doesn't add any parameter details beyond implying the token is a JWT. It doesn't explain format requirements (e.g., string encoding) or constraints. With low schema coverage, the description fails to compensate adequately, but the single parameter is straightforward, so it meets the baseline for minimal viability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Analyze a JWT for common vulnerabilities and issues.' It specifies the verb ('analyze') and resource ('JWT'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like jwt_decode (which might decode without analysis) or jwt_bruteforce (which might attempt exploitation), leaving some ambiguity about its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools (jwt_bruteforce, jwt_decode, jwt_generate) or specify contexts like security testing versus debugging. Without any usage context, the agent must infer based on tool names alone, which is insufficient for clear decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jwt_bruteforceA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the brute-force method and wordlist options, it doesn't disclose critical behavioral traits like computational intensity, time requirements, potential rate limiting, ethical considerations, or what happens when a secret is found. For a security testing tool with zero annotation coverage, this represents significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with a single sentence that front-loads the core purpose. Every word earns its place with no wasted language, making it immediately understandable while covering the essential what and how.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a security testing tool with no annotations, no output schema, and 0% schema description coverage, the description is insufficiently complete. It doesn't explain what happens when the tool succeeds or fails, what output to expect, error conditions, or important constraints. For a tool performing cryptographic attacks, more contextual information is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for both parameters, the description must compensate but only partially succeeds. It mentions 'token' and 'wordlist' but doesn't explain what format the JWT token should be in, what constitutes a valid wordlist, or how the wordlist array should be structured. The description adds minimal semantic value beyond parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Bruteforce the secret') and target resource ('for HS256/HS384/HS512 JWTs'), distinguishing it from siblings like jwt_analyze, jwt_decode, and jwt_generate. It precisely communicates the cryptographic attack method and JWT algorithm targets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (for brute-forcing JWT secrets with specific algorithms), but doesn't explicitly state when NOT to use it or mention alternatives. It implies usage for security testing scenarios but lacks explicit exclusions or comparison with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jwt_decodeA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool decodes without verification, which is a key behavioral trait. However, it lacks details on error handling, output format, or any rate limits or permissions needed, leaving gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and key limitation ('no verification'), with zero wasted words, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (decoding without verification), no annotations, and no output schema, the description is adequate but incomplete. It covers the basic purpose and a key limitation, but lacks details on return values, error cases, or how it differs from siblings, which could hinder agent selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema by specifying that the 'token' parameter is a JWT to decode. Since schema description coverage is 0% and there is only one parameter, this compensates well, though it could provide more details on token format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Decode a JWT') and the resource (a JWT token), distinguishing it from siblings like jwt_analyze, jwt_bruteforce, and jwt_generate by specifying it returns header, payload, and signature without verification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating 'no verification,' implying this tool is for inspection rather than security validation. However, it does not explicitly mention when to use it versus alternatives like jwt_analyze or jwt_bruteforce, which might offer more detailed analysis or verification features.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.