Server Quality Checklist

Tool Scores

• check_vtilA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that WARN (not ERROR) is reported when binary is missing, and details the fallback chain for finding the binary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: three lines, front-loaded with the main purpose, then behavioral details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description is complete in explaining the tool's functionality and fallback behavior. Could slightly improve by noting it is a health check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so schema coverage is 100%. The description adds no parameter info, but none is needed. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Return vtil-cli version + supported architectures,' which is a specific verb+resource. It distinguishes itself from siblings like lift_handler and optimize, which are for code manipulation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is given. The description implies the tool is for checking the environment, but does not mention alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lift_handlerA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It discloses the output structure and an error case for missing binary, but does not mention side effects, safety, or non-destructive nature, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns sections, and each sentence adds value. It is slightly verbose but not wasteful; front-loads the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 params, no output schema, no annotations), the description covers input details, output structure, error handling, and implies a workflow with siblings. It is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description thoroughly explains each parameter: 'arch' with enumerated values, 'code' with base64 encoding rationale, and 'base_address' with default and context. This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action ('Lift') and the resources ('arch' and 'code' to VTIL IL), making the purpose unambiguous. It distinguishes from siblings like 'optimize' and 'emit' by being the entry point for lifting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (output is usable by 'optimize' and 'emit') but does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• simplify_lifted_ilA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description is sole source. It describes passes applied and return value but does not discuss side effects, error handling, or destructive nature. Moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections. Front-loaded with purpose. Every sentence adds value. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema or annotations, description covers parameters and return format adequately. However, missing details on error handling and input validation. Fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% but description adds context: il is from lift_handler, passes can be None, default_preset options are given. Adds meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a curated default pass set on a lifted IL tree, specifying the use case for encrypted-VM bytecode handler-lift. It distinguishes from siblings by naming specific pass set and referencing lift_handler output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it should be used after lift_handler but does not explicitly state when to use it vs alternatives like 'optimize' or 'check_vtil'. No when-not or alternative tools mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• emit_pseudo_cA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It transparently states the output is best-effort and low quality, and explicitly lists the return format (code and il_block_count). No side effects or auth needs are mentioned, but these are not expected for a read-only emit function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear purpose statement, followed by a brief explanation of quality and use case, then structured Args and Returns sections. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool with one parameter and no output schema, the description covers the input source and output format. However, since the schema coverage is 0% and the parameter is a nested object, more detail on the il object's expected structure would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, il, is explained as the IL tree (raw from lift_handler or optimized via optimize). This adds value beyond the schema, which only specifies type: object. The context covers the parameter's origin and variants, though it does not detail expected keys or structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: emit a pseudo-C reading of a lifted IL tree. The verb 'emit' and resource 'pseudo-C reading' are specific, and the tool is distinct from siblings like lift_handler or optimize, which focus on different stages of the pipeline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use the tool (quick first-pass read of VM handler body) and notes its limitations compared to IDA Hex-Rays or Ghidra. It also hints that the input can come from lift_handler or be optimized via optimize, but does not explicitly exclude other uses or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• optimizeA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns an optimized IL with a list of applied passes, and that the output shape matches input. It does not mention whether the input is mutated or side effects, but the context implies non-destructive optimization.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear docstring format: a single sentence for purpose, then Args and Returns sections. Every sentence adds value, with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters, nested object, no output schema), the description covers all necessary aspects: input source, pass selection, return format, and relation to sibling tools. It is complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema: 'il' is described as 'the IL tree produced by lift_handler', and 'passes' explains the canonical set and that pass names are C++ enum names. This compensates for 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Run VTIL optimization passes over a lifted IL tree.' It includes a specific verb and resource, and distinguishes from siblings like 'lift_handler' and 'emit_pseudo_c' by focusing on optimization passes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the input (IL tree from lift_handler) and the canonical passes, plus that the output is a drop-in replacement for downstream emit calls. It doesn't explicitly compare to alternatives like 'simplify_lifted_il' but provides sufficient context for proper usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.