Server Quality Checklist

Tool Scores

• solana_forensics_executeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that large results are automatically paginated and that continue is needed for subsequent pages, adding useful behavioral context beyond annotations. However, it does not mention whether operations are read-only, any side effects, authentication requirements, or error behavior, which is important for a generic executor tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Highly concise and well-structured: purpose, prerequisite, argument explanation, and pagination are each addressed in separate sentences without redundancy. Every sentence serves a clear function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the generic nature of the tool, the description covers the essential workflow: get schema, execute, handle pagination. The presence of an output schema reduces the need to describe return values. It could improve by explicitly mentioning discover for listing operations, but the sibling tool names imply that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description compensates partially by explaining the operation parameter as an identifier (with an example) and the params parameter as matching the schema. Yet it does not enumerate possible operations or provide details about the params structure beyond referencing the schema, leaving significant ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it executes a Solana forensics operation with specified parameters, providing a concrete example operation identifier. It distinguishes itself from siblings by being the executor, while discover, get_schema, and continue serve auxiliary roles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use solana_forensics_get_schema first to understand required parameters, and mentions solana_forensics_continue for pagination. However, it does not explicitly mention when not to use this tool or mention solana_forensics_discover for finding available operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solana_forensics_get_schemaA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It states that it returns parameter requirements, types, and descriptions, and uses 'Get' which implies read-only behavior. However, it doesn't explicitly declare side-effect-free status, error handling, or prerequisites beyond knowing the operation identifier.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-line purpose, an Args section with a clear parameter explanation, and a return summary. Every sentence provides value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter introspection tool, the description is mostly complete. It covers the operation format and return value, and since an output schema exists, return details don't need to be over-explained. A minor gap is not mentioning that discover can list valid operations, but this is an acceptable omission given sibling context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage for the 'operation' parameter, but the description compensates by giving the format 'category.action' with concrete examples. This adds significant meaning beyond the schema, though it could further mention where to discover valid operations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get detailed parameter schema for a specific operation' with a specific verb and resource. It provides the operation format and examples, distinguishing it from sibling tools like execute and discover.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for when you need the parameter schema of a specific operation, but it doesn't explicitly state when to use it versus alternatives or mention exclusions. No reference to sibling tools like discover for enumeration or execute for running operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solana_forensics_discoverA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It states that the tool returns a tree of operations with descriptions, which is useful. However, it does not explicitly confirm read-only behavior or mention any potential side effects, rate limits, or required permissions. For a simple discovery tool, this is acceptable but could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loaded with the main purpose, and every sentence adds value: what it does, what it returns, and when to use it. There is no wasted wording or redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, discovery-oriented) and the presence of an output schema, the description is sufficiently complete. It explains the purpose, output, and usage context. It could mention specific categories or that it is safe to call anytime, but these are minor gaps. Overall it provides enough context for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. Per the rubric, a baseline of 4 is appropriate when there are no params to explain. The description adds nothing about parameters but does not need to. It provides high-level context about the output structure, which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb 'browse' and resource 'Solana forensics operations organized by category'. It distinguishes itself from sibling tools like execute and get_schema by describing an exploratory/discovery function, making its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this to explore what operations are available before executing them', providing clear when-to-use guidance. It does not explicitly name alternative tools, but the instruction to use it before executing implies when not to use it (when ready to execute, use execute). This is clear but lacks a direct contrast to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solana_forensics_continueA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explains the core behavior and session_id default, but doesn't disclose potential errors (e.g., no previous operation), read-only nature, or side effects. Adequate but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with the purpose front-loaded and the parameter explanation following. No redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with an output schema present, the description covers the essential context: what it does and how the session_id works. It could be slightly more explicit about prerequisites, but it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description fully explains the only parameter session_id: its source and the fallback to the last paginated operation. This adds complete meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Continue retrieving paginated results from a previous operation' with a clear verb (continue) and resource (paginated results), clearly distinguishing itself from sibling tools like discover, get_schema, and execute.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly implies when to use (after a paginated operation) and explains the default behavior when session_id is omitted. However, it doesn't explicitly mention alternatives or exclusions, though there are none relevant.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.