Server Quality Checklist

Tool Scores

• deregister_interactsh_sessionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states the tool removes a session from both interactsh and local cache, implying a destructive operation, but doesn't clarify if this is irreversible, requires specific permissions, or has side effects (e.g., affecting ongoing interactions). More details on consequences or limitations are needed for a higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the core action and efficiently conveys the essential information without unnecessary elaboration, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's destructive nature (implied by 'Removes'), lack of annotations, no output schema, and incomplete parameter documentation, the description is insufficient. It doesn't address what happens post-deregistration, error conditions, or provide enough context for safe and effective use, leaving significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description doesn't mention the parameter 'correlation_id', and schema description coverage is 0%, so it adds no semantic information beyond the schema. However, with only one parameter, the baseline is 3, as the schema alone might suffice for a simple input, but the description fails to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Removes') and the target ('a session from interactsh and local cache'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_interactsh_sessions' or 'poll_interactsh_session' beyond the obvious action difference, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For example, it doesn't mention prerequisites (e.g., requiring an existing session) or warn against misuse (e.g., deregistering active sessions). The description lacks context about its role relative to siblings like 'create_interactsh_session'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_interactsh_sessionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It mentions generating credentials and registering, which implies a write operation, but lacks details on permissions, rate limits, or what happens if registration fails. This is a mutation tool with significant behavioral gaps in disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key actions without unnecessary words. Every part ('generates credentials', 'registers', 'returns connection details') contributes directly to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what 'connection details' include, error handling, or dependencies on other tools like 'deregister_interactsh_session', leaving gaps in contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately adds no parameter details, focusing on the tool's action, which aligns with the baseline for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('generates credentials', 'registers', 'returns connection details') and identifies the resource ('interactsh session'). It distinguishes from siblings like 'deregister_interactsh_session' and 'list_interactsh_sessions' by focusing on creation, though it doesn't explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies usage for creating a new session but doesn't mention prerequisites, when not to use it, or refer to sibling tools like 'poll_interactsh_session' for related actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_interactsh_sessionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that sessions are 'cached in memory', which adds some context about data persistence, but fails to disclose key behavioral traits like whether this is a read-only operation, potential rate limits, or the format of the returned list. The description is too minimal for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without any wasted words. It is front-loaded and appropriately sized for a simple listing tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. However, it lacks details on output format or behavioral context, which could be important for an agent to use it correctly. It meets the basic requirement but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description does not add parameter details, which is appropriate, but it implies the tool operates on cached data without requiring inputs, aligning well with the schema. A baseline of 4 is set for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Lists') and resource ('interactsh sessions cached in memory'), making the purpose unambiguous. However, it does not explicitly differentiate from sibling tools like 'poll_interactsh_session', which might also involve listing or retrieving session data, so it doesn't fully achieve sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'poll_interactsh_session' or 'create_interactsh_session'. It lacks context about prerequisites, such as whether sessions need to be created first, or exclusions for when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• poll_interactsh_sessionB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'retrieves and decrypts' which implies read-only behavior, but doesn't address critical aspects like authentication requirements, rate limits, error conditions, or what 'decrypts' entails operationally. For a tool with 6 parameters and no annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and well-structured in just two sentences. The first sentence states the core purpose, and the second explains the filtering capabilities. Every word earns its place with zero redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (6 parameters, 1 required), no annotations, and no output schema, the description provides adequate but incomplete context. It covers the purpose and parameter semantics well but lacks behavioral details and usage guidelines. For a tool that 'decrypts' data and has multiple siblings, more comprehensive guidance would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides excellent parameter semantics despite 0% schema description coverage. It clearly explains that 'optional filters let you match HTTP method, path, query, protocol, or free text,' which maps directly to 5 of the 6 parameters (method, path_contains, query_contains, protocol, text_contains). Only 'correlation_id' isn't explicitly mentioned, but its purpose is implied by 'for a session.' This effectively compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('retrieves and decrypts') and resource ('interactions for a session'), making it easy to understand what the tool does. However, it doesn't explicitly distinguish this polling tool from its sibling tools (create, deregister, list sessions), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'optional filters' but provides no guidance on when to use this tool versus its siblings (create_interactsh_session, deregister_interactsh_session, list_interactsh_sessions). There's no indication of prerequisites, sequencing, or alternative scenarios, leaving the agent with minimal context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.