Server Quality Checklist

Tool Scores

• get_ida_database_infoA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, destructiveHint, and idempotentHint, covering safety and idempotency. The description adds the return format (JSON string) but does not disclose other behavioral traits beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with three short sentences covering purpose, parameters, and returns. No redundant information; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only information tool with strong annotations, the description is complete: it specifies the action, parameter semantics, and return format. The sibling context provides differentiation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds necessary meaning: it explains that instance_id is 'Target IDA instance ID (optional, uses default if not specified)', clarifying its role and default behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get information about an IDA database', specifying the verb and resource. It distinguishes itself from sibling tools like execute_idapython or list_ida_instances, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (to get database info) and mentions the optional instance_id, but lacks explicit guidance on when not to use it or how it differs from alternatives like check_ida_connection or check_task_status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ida_instancesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, idempotent, and open-world behavior. The description adds valuable context by specifying that only running instances with the MCP plugin are listed and provides the return format with an example, including count and instance details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: one sentence for purpose, then return format and example. Every sentence adds value, no redundancy. Front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, clear purpose, and output schema described in the description), the description is complete. It covers what the tool does, what it returns, and provides a concrete example. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema is fully covered. Per guidelines, 0 parameters yields a baseline of 4. The description adds no parameter info, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool lists all running IDA instances with the MCP plugin enabled, specifying the verb 'List' and the resource 'running IDA instances'. It implicitly distinguishes from sibling tools like 'execute_idapython' and 'check_ida_connection', which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. While the purpose is clear, there is no mention of when not to use it or which sibling tool might be more appropriate for specific scenarios, such as checking a single connection or executing scripts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_ida_connectionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true. Description adds that it returns a JSON-formatted string with connection status and instances, providing return format context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two effective sentences: one for purpose, one for return. Front-loaded, no filler, every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple (0 params, output schema exists). Description adequately covers purpose and return format for a connection check, though could mention lightweight nature given annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Zero parameters with 100% schema coverage, so baseline is 4. Description does not need to add param info and appropriately omits it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states specific verb+resource ('Check if IDA Pro instances are running and accessible'), clearly distinguishing it from siblings like list_ida_instances which may only list without checking accessibility.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this vs alternatives (e.g., list_ida_instances). The purpose implies it's for verifying connectivity, but lacks when-not-to-use or condition-based selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_idapythonA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (destructiveHint: true), the description details execution in IDA's main thread, polling with timeout, and returning a task_id for async status checks. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with clear sections (summary, execution behavior, Args, Returns, Examples). It is somewhat verbose but all content is necessary for a complex tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all aspects: execution context, timeout handling, return format (result, stdout, stderr, instance, task_id), and examples. No gaps given the tool's complexity and available annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite schema having partial descriptions, the description's Args block adds valuable context: default values, the mutual exclusivity of code and script_path, and timeout behavior. Examples illustrate parameter use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute Python code or script file in IDA Pro context.' It distinguishes from sibling tools like check_ida_connection and check_task_status by focusing on code execution with explicit examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use (to execute code in IDA) and when to use the sibling check_task_status tool for checking long-running scripts. Provides examples for default and specific instances, guiding the agent on proper usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_task_statusA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds behavioral context by explaining the tool's role in polling a long-running task and specifying the return format (JSON status with result). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise (4 sentences) with purpose front-loaded. Every sentence adds value, no fluff. Structure includes clear Args section and Return description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description adequately covers usage, parameters, and return. It references the related execute_idapython tool, providing necessary context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description provides full parameter descriptions: task_id is 'The task ID returned by execute_idapython', instance_id and port are optional with defaults explained. This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check the status of a long-running script execution task.' It specifies the verb (check) and resource (task status), and distinguishes from siblings by referencing the task_id from execute_idapython.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'when execute_idapython returns a task_id with status running or pending.' This provides clear context and implicitly tells when not to use (e.g., if status is completed).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.