Server Quality Checklist

Tool Scores

• run_pythonA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses pre-bound namespace objects, return value semantics (trailing expression or result variable), stdout capture, and truncation. However, it does not warn about the potential side effects of running arbitrary code in the server process, which is a significant omission for a code execution tool. With no annotations, the description carries full burden, so this gap caps the score at 3.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short, purposeful paragraphs with no filler. Purpose, usage guidance, and behavioral details are each covered efficiently, front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, return behavior, and truncation, but omits error handling (e.g., what happens on exceptions), state persistence across calls, and side-effect warnings. For a complex and potentially destructive tool like arbitrary code execution, these are notable gaps, making the description not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'code' has no schema description (0% coverage). The description adds context by explaining the execution environment (pre-bound namespace) and return behavior, but it does not explicitly describe what the code parameter should contain in terms of format, examples, or constraints. It adds some meaning but not enough to fully compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Run DaVinci Resolve scripting-API Python in the server process' with a specific verb and resource. It also distinguishes itself from sibling project management tools by positioning as a fallback for API corners not covered by the catalog.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given: 'Prefer the real tools' and 'Reach for this only where no tool fits.' Specific use cases are named—'an API corner the catalog does not cover yet, or a one-off inspection'—clearly delineating when to use this vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• snapshot_projectA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses key behaviors: the backup is opaque, and without a path it goes to a timestamped file in the cache directory. It does not cover potential errors or side effects, but the main behavior is well explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear one-sentence action followed by a usage scenario paragraph. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (one optional param), the description adequately covers purpose, default behavior, and usage scenarios. An output schema exists, so detailed return value explanation is not required. It lacks information about failure modes but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the single parameter, but the description explains that without a path, the snapshot goes to a timestamped cache file, giving meaning to the optional path parameter. It could be more explicit about what path should be, but it effectively compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool writes an opaque .drp backup of the open project and returns the location. It distinguishes itself from siblings like list_projects and open_project by focusing on creating a backup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to use: 'before any big operation' such as builds, bulk media changes, or risky scripts. It does not explicitly mention when not to use or name alternatives, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the output behavior—echoing context with fields like connected, version, project, timeline, and fps. It does not explicitly state the tool is read-only, but the nature of a status report implies safe, non-mutating behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the purpose, and each sentence adds value: what it reports, when to call it, and what the result contains. No redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple status tool with no parameters and no annotations. The description explains the purpose, usage timing, and output content. An output schema exists, but the description already covers the returned fields, so it is complete for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so the input schema is empty. The description adds no parameter-level details, but the baseline for 0 params is 4, and the description sufficiently explains what the tool reports.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Report' and identifies the resource: connection state, Resolve version, project, and timeline. This clearly distinguishes it from sibling tools like list_projects and open_project.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call this first in a session, and again whenever you are unsure what Resolve has open.' This provides clear when-to-use context, though it does not mention when not to use it or alternatives, which is less critical for a status check.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_projectA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It clearly discloses the key side effect (all later calls act on the opened project) and the output behavior (echoes the new context for confirmation). It does not mention error behavior or reversibility, but the most critical behavioral trait is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences deliver the core purpose, prerequisites, and output behavior with no filler. The main action is front-loaded, and every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter context-switching tool, the description covers the use case, how to get valid input, and how to verify success. An output schema exists, so return details need not be spelled out. The description is sufficient without being verbose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines 'name' as a required string with no description. The description adds essential semantics: the name must match exactly, and list_projects provides the available names. This fully compensates for the 0% schema coverage and makes the parameter's meaning unambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Open') and resource ('named project') and immediately clarifies the tool's actual function: making the project the target of every later tool call. This distinguishes it from siblings like list_projects and snapshot_project, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the name must match exactly and points to list_projects as the way to discover valid names, providing clear prerequisite context. It does not explicitly contrast with other sibling tools, but the context-switching purpose is clear enough for an agent to know when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that it lists names from the current database folder and adds value by noting the exact-match requirement. The verb 'list' implies read-only, but it does not explicitly state what happens if no projects exist or whether it might be slow. Still, it provides sufficient transparency for this simple operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences convey the purpose and key usage guidance without any wasted words. Every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no complex behavior) and the presence of an output schema, the description fully covers what the agent needs: purpose and how to use it in context with open_project. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so baseline is 4. No parameter descriptions are needed, and the description adds no irrelevant parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it lists project names in Resolve's current database folder, using the specific verb 'List' with a defined resource. This distinguishes it from siblings like open_project, which performs a different action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use it: to find the exact name to pass to open_project, emphasizing exact match. This provides clear contextual guidance and implies not using it for other purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.