Server Quality Checklist

Tool Scores

• search_toolsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden but only implies it is a read operation. It does not state whether results are paginated, whether limit caps returned matches, how matching is performed (tag/query/method combined or any-of), or what the result payload looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with zero waste. Nothing redundant or padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter catalog-search tool with no annotations and no output schema, the description is too thin: it omits filtering semantics, result shape, and pagination behavior, leaving the agent to guess at how to construct a useful query.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description mentions none of the four parameters (tag, limit, query, method). The agent cannot tell from either source whether these filters are ANDed, what 'tag' refers to, or that limit defaults to 100 and maxes at 100 — a notable gap for a search tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (Search) and resource (all operations in the official Frame.io V4 OpenAPI catalog), which is more precise than a bare 'search'. It does not name or differentiate itself from siblings such as get_tool_schema or invoke_tool, so the agent must infer the discovery-to-invocation workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description never says when to use this tool versus the sibling tools (get_tool_schema, invoke_tool) that operate on the same catalog. No prerequisites, no exclusions, no statement of what to do with the results.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• upload_local_fileC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, so the description carries the full burden. It hints at the mechanism (presigned URLs, implying a direct-to-storage transfer) and that it returns upload status, but omits required permissions/auth, file size limits, overwrite behavior, and whether failures are partial or atomic — all material for an upload tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single tight sentence with the verb front-loaded and no filler. Nothing redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, no output schema, and 4 undocumented parameters, the description is too thin. It should disclose permission needs, size/type constraints, and what 'upload status' entails, none of which are present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% across 4 parameters (file_path, account_id, folder_id, name). The description only loosely implies file_path is a local path; account_id, folder_id, and the optional name are never explained. With zero schema coverage, the description should have compensated but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (upload), resource (local file), and mechanism (Frame.io V4 presigned URLs), plus what it returns (upload status). An agent can tell what this does without opening the schema. Sibling tools are unrelated meta-tools, so no differentiation is required.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use guidance, no prerequisites, no alternatives, and no indication of when this is appropriate versus not. The agent gets a capability statement but zero routing context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoke_toolC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses one useful trait (the confirm requirement for DELETE), but says nothing about side effects, whether arbitrary operations are destructive, authentication needs, or how the confirm gate is enforced for other write verbs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler, and the constraining rule is front-loaded after the core purpose. It is efficient, though arguably too terse given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a high-complexity generic gateway with five under-documented parameters, nested objects, no annotations, and no output schema, yet the description gives no guidance on constructing path/query/body payloads or discovering operation_ids. For a tool of this scope the description is materially incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% across five parameters, including nested body/path/query objects that carry no structure or examples. The description only hints at the semantics of 'confirm' (via the DELETE note) and says nothing about operation_id, body, path, or query, so it fails to compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Invoke') and resource ('one operation from the official Frame.io V4 OpenAPI catalog'), so an agent can tell it performs a passthrough API call. It does not differentiate itself from siblings such as get_tool_schema or upload_local_file, which also touch the tool surface, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It includes one concrete usage rule (DELETE requires confirm=true), which implies when the confirm parameter is mandatory. However, it never states when to prefer this generic invoker over siblings like upload_local_file or how to obtain a valid operation_id, leaving most selection guidance to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tool_schemaB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It implies a read-only lookup but never states that it is non-mutating, what happens with an unknown operation_id, or whether the returned schema is authoritative. The word 'official' hints that returned parameters should be used verbatim, which is modest added value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with zero filler, stating the resource, return content, and scope. Nothing is redundant or buried.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description only loosely sketches the return ('parameters and referenced schemas') and omits the workflow context — that this lookup precedes invocation — and error behavior for invalid ids. Adequate but with clear gaps for a schema-discovery tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the sole required parameter operation_id is undocumented in the schema. The description adds partial semantics by identifying it as naming 'one Frame.io V4 operation', but gives no format, source, or validation detail, so it does not fully compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description gives a specific verb and resource ('Get official parameters and referenced schemas') scoped to 'one Frame.io V4 operation', which is more precise than the bare name get_tool_schema. It does not, however, explicitly distinguish itself from siblings like search_tools or invoke_tool, leaving the agent to infer its place in the toolchain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no when-to-use guidance, no stated prerequisite (e.g., that operation_id typically comes from search_tools), and no reference to the sibling tools. An agent must guess that this is a discovery step to run before invoke_tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_connectionA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It does disclose the underlying HTTP call (GET /v4/me), which tells the agent the operation is a read with no mutation, but it says nothing about what success vs. failure looks like, required credentials/token state, or error/rate-limit behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with the purpose first and the mechanism second; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless connectivity check with no output schema, the description covers what it does and how it does it. It could say more about interpreting the result (success/failure semantics, auth prerequisites), but nothing essential is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so per the baseline rule a 4 applies. There is nothing for the description to disambiguate beyond what the empty schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (verify) and a specific resource (Frame.io V4 OAuth), plus the exact mechanism (GET /v4/me), so the agent knows precisely what the tool does. It is unambiguous against the unrelated meta siblings, though it does not explicitly contrast itself with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is only implied: an agent can infer this is a pre-flight auth/connectivity check, but there is no explicit 'use before X', 'do not use when Y', or named alternative. Adequate but with a clear gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.