Server Quality Checklist

Tool Scores

• recommend_tierB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses it combines a preflight scan with a sensitive-string-literal heuristic and returns concrete actions. No annotations exist, so description carries the burden; it could mention if it modifies anything or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with purpose, method, and output. Efficient but could be more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, method, and output types (free_action, pro_action), but lacks usage guidelines and parameter details. Adequate for a simple tool with one parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter 'path' has no description in the schema (0% coverage) and the tool description only implies it's the project path without explicit semantics or format details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it analyzes a Python project and recommends a tier (community vs Pro) with reasons, using a heuristic. However, it does not explicitly differentiate from sibling tools like check_obfuscation_risks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when a tier recommendation is needed, but no explicit when-not or alternatives to sibling tools are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_obfuscation_risksA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden. It implies a read-only operation via 'scan' and lists return values, but does not explicitly state that the tool does not modify files, execute the project, or require special permissions. It also does not mention any limitations or edge cases beyond the listed patterns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and front-loads the main purpose. The first sentence lists example patterns, the second lists return items. Every word adds value, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and single parameter, the description is reasonably complete: it states what it scans, what it returns, and includes a suggested preset. However, it lacks context on when to invoke it in a workflow and does not explain how to interpret the results. With an output schema present, the return values are partially covered, so the description covers the essentials.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single 'path' parameter with no description, but the tool description says 'Scan a Python project,' clarifying that 'path' refers to the project location. This adds meaning beyond the raw schema, though it does not specify whether path should be a directory, file, or path format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans a Python project for patterns that may break obfuscation, listing specific examples (eval/exec, dynamic attribute access, etc.). It also specifies the output (severity counts, frameworks, suggested preset), making it distinct from sibling tools that generate configs or obfuscate projects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives like protect_project or recommend_tier. There is no mention of prerequisites, whether to run before obfuscation, or exclusions for other tools. Usage is only implied by the verb 'scan'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unmap_stack_traceA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden for behavioral transparency. It does not disclose whether the tool is read-only, requires any permissions, has side effects, or error handling behavior. The description only states the action and inputs, leaving gaps about what happens internally or under failure conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two sentences that deliver all necessary information without fluff. It is front-loaded with the action and efficiently explains inputs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two simple parameters and an output schema, the description is fairly complete. It explains the purpose, inputs, and the origin of the mapping file. It could briefly mention what the output looks like (e.g., unmapped trace), but the output schema handles that. Overall, it provides sufficient context for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description effectively compensates by naming both parameters and providing context: 'trace' is described as 'plain text', and 'mapping_path' is described as 'the path to a mapping file produced by --save-mapping.' This adds meaningful usage guidance beyond the bare schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Reverse obfuscated identifiers in a stack trace using a pyobfus mapping.json.' It specifies the verb (reverse), resource (stack trace), and method (mapping file), which distinguishes it from sibling tools like check_obfuscation_risks and generate_pyobfus_config.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating what it does and what inputs are needed (trace as plain text, mapping file path). However, it does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or when-not-to-use guidance. Given the sibling tools, usage context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_presetA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds useful context by specifying that Pro presets return 'full pro_unlock metadata (trial command, checkout URL, price, money-back guarantee)', which goes beyond the bare tool name. It does not explicitly state read-only behavior, but the term 'explain' and the nature of the described outputs make non-mutation apparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence front-loads the primary purpose and key covered aspects, while the second adds a meaningful conditional detail about Pro presets. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose, key features, and special behavior for Pro presets, which is sufficient given a single simple parameter and the presence of an output schema. It could mention a usage workflow (e.g., 'use after list_presets') but is not incomplete without it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It implicitly identifies the sole 'name' parameter as a preset name through the phrase 'named preset' and adds meaning by distinguishing Pro presets. However, it does not provide explicit parameter documentation, such as example names or format, leaving some room for interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Describe' and the resource 'a named preset', and lists specific aspects ('exclude names count, exclude patterns, preserve_param_names, docstring handling') that differentiate it from siblings like list_presets. It is immediately clear what this tool accomplishes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides enough context to infer the tool is used for explaining preset details, and the presence of list_presets implies an alternative for listing. However, there is no explicit 'when to use' or 'when not to use' guidance, nor a direct reference to sibling tools, leaving it to the agent to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_pyobfus_configA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  It discloses that it auto-detects frameworks and the default vs. write behavior, which is important for a generation tool, though it doesn't specify error handling or file overwrite policy.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loading the purpose and then providing key behavioral details without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the main action and default behavior, it omits details on the 'preset_override' parameter and error scenarios, making it adequate but not thorough for a tool with auto-detection logic.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description only explains the 'write' parameter; 'path' and 'preset_override' are not described despite 0% schema coverage, leaving the agent unclear on expected values and semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a pyobfus.yaml for a Python project, which is specific and distinguishes it from siblings like protect_project or recommend_tier.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the default behavior (returns YAML text) and when to use the write parameter, but does not explicitly state when not to use it or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• protect_projectA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It details the entire workflow: scans risks, picks a preset, obfuscates, byte-compiles, runs import-smoke-tests in isolated subprocesses, and returns verified:true/false. It also mentions writing a private de-obfuscation mapping alongside the output. It does not specify permissions or failure modes, which is a minor gap, but overall it provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long with no filler. The first sentence is a dense, front-loaded summary of the entire workflow. The second adds a key behavioral detail (mapping placement). The third provides concise usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 params, 1 required, output schema present) and 0% schema coverage, the description covers the core workflow and usage well. It explains the main steps and expected outcome. It does not detail the output schema or all parameter behaviors, but the output schema likely handles return value documentation. Slight gap in parameter descriptions, but overall complete enough for an AI to determine appropriate invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It provides high-level context for some parameters (e.g., 'path' as project path, 'preset' as framework-aware, 'verify' as the verification step, 'save_mapping' for the mapping file), but it does not explicitly detail all 8 parameters. For example, 'timeout', 'output_dir', 'verify_cmd', and 'trace_marker' are only briefly implied. This is minimally adequate but not thorough.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'protect a Python project end-to-end AND verify it still works'. It distinguishes itself from sibling tools by emphasizing it is a combined protect+verify operation that 'returns verified:true/false', not just a transform. The context of sibling tools like check_obfuscation_risks and generate_pyobfus_config reinforces that this is a higher-level, end-to-end tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use: 'Use this when the user wants to protect/obfuscate before shipping and expects a green check, not just a transform.' While it doesn't name specific alternatives, the sibling list implies that other tools are for more granular steps (e.g., risk scanning or preset generation). The guidance is clear but could be more explicit about when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_presetsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden. It discloses the listing behavior and the grouping structure via 'grouped by tier'. The verb 'List' implies a read-only operation, and since an output schema exists, the return format need not be detailed further.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the verb 'List' and directly states the scope and grouping. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no params and an output schema, the description fully answers what the tool does. It covers the listing and the grouping, and there are no preconditions or side effects to mention.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so schema coverage is trivially 100%. The description adds no parameter information because none is needed; the baseline for 0 params is 4, and there is nothing to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every pyobfus preset and groups them by tier (community / framework-aware / Pro). This is a specific verb+resource+scope, and it differentiates from siblings like explain_preset and recommend_tier.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case of discovering all available presets, which is clear enough from 'List every'. It does not explicitly name alternatives or exclusions, but the context of sibling tools (e.g., explain_preset) makes it obvious when this tool is the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_pro_trialA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully conveys behavioral traits: it returns structured guidance, does not invoke side effects, detects active trials, and surfaces next steps plus purchase URL. This is transparent and leaves no ambiguity about what the tool does or does not do.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. The first sentence states purpose and critical side-effect disclaimer; the second adds behavioral detail. Highly efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no annotations, and the presence of an output schema, the description is complete. It covers purpose, non-side-effect nature, detection capability, and what guidance is returned. No missing information for an agent to correctly select and invoke this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100%, so baseline is 4. The description does not need to add parameter information; it implicitly covers the lack of parameters by describing the tool's function without mentioning inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: return structured guidance for starting the 5-day Pro trial. It specifies the resource ('pyobfus Pro trial') and action ('return structured guidance'), and distinguishes itself from sibling tools by noting it does not invoke the side effect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: it is for guidance only, not for actual trial activation, as the user must run `pyobfus-trial start` in their shell. It also mentions detecting active trial status, which helps an agent decide whether to call this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.