Server Quality Checklist

Tool Scores

• disconnectA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing side effects. It only states the action without mentioning whether it is destructive, requires authentication, or what happens if there is no active backend. This is insufficient for a mutating operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence (7 words), front-loads the verb, and contains no unnecessary text. It is perfectly sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for basic understanding but lacks usage context (when to call) and behavioral details (side effects). While the tool is simple and output schema exists, the lack of guidance reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema trivially covers all inputs. The description adds no parameter details, but none are necessary; baseline 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action ('Disconnect') and the target resource ('cfx leaf's active backend'), which is specific and distinguishes it from sibling tool 'connect' as the inverse operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs alternatives, such as prerequisites, whether to call after 'connect', or behavior when no active backend exists. This is a clear gap for a tool with a direct counterpart.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cfx_workflowA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits, but it only lists action names without explaining side effects, blocking behavior, or return values. For instance, it does not clarify that wait_solver blocks until completion or that start_solver may launch background processes, leaving the agent underinformed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences with no redundant phrasing. The list of actions is compact and the sentence about the external agent layer provides important exclusion guidance without wasting words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema (not shown) and eight distinct actions, the description is sparse. It does not detail what each action does, what parameters are required per action, or what output to expect. For a tool with such varied behavior, an agent would struggle to invoke actions like write_def or get_results_file correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes both parameters, so the baseline is 3. The description adds significant value by enumerating the allowed action values (start_pre, import_mesh, etc.), which are not present as an enum in the schema. It also gives concrete examples (file paths, launch options, timeout settings) that clarify the flexible 'params' object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Run') and resource ('CFX lifecycle or artifact action'), and it enumerates the exact actions (start_pre, import_mesh, etc.). This distinguishes it well from sibling tools like run_code and validate_code, which focus on Python code execution rather than predefined CFX operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (for focused CFX lifecycle/artifact actions) and explicitly directs users to the external agent layer for custom PyCFX code generation, which is a useful exclusion. However, it does not explicitly compare against siblings like run_code or session_status, so the guidance is not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cfx_model_contextA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that responses are 'targeted, compact' and that max_items limits size, but does not state whether the tool is read-only, possible errors, or any side effects. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose, followed by a compact action list and max_items guidance. No redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has many actions and parameters, the description covers the core purpose and output control but lacks guidance on when to choose this tool over siblings or how to select between actions. Output schema covers return values, so no need to explain them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by enumerating valid action values and explaining the purpose of max_items, which goes beyond the schema's generic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and resource ('CFX model context slice'), and enumerates distinct actions. It clearly distinguishes this as a context-query tool from siblings like run_code or connect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (for context queries) and lists actions, but does not explicitly state when not to use it or mention alternatives among sibling tools. No exclusionary or comparative guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connectA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses backend kinds, auto-selection behavior, and kwargs forwarding, but it doesn't mention side effects such as whether an existing connection is replaced, whether authentication is needed, or error behavior. This is moderate transparency with notable gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences lead with the core purpose, then cover backend selection and option forwarding. No filler or redundant restatement of schema fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two optional parameters, an output schema, and full schema coverage. The description sufficiently explains both parameters and the overall connect workflow, though it could briefly note prerequisites or state implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3, but the description adds meaning: it lists the available backend kind ('pycfx'), explains that omitting it triggers auto-selection, and clarifies that 'connect_kwargs' is a dict of forwarded backend-specific options like url/token/ip/port. This adds real value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Connect the cfx leaf to a backend,' naming both the resource and the target. It also distinguishes this from sibling tools like disconnect and session_status by focusing on establishing a connection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete usage direction: choose 'backend_kind' or omit to auto-select, and put backend-specific options in 'connect_kwargs'. It doesn't explicitly mention when not to use it, but the action is contextual and obvious relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_codeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully carries the behavioral burden. It discloses that it is a dry-run and has no side effects, which is critical. It also states it returns parse/type/semantic feedback, offering insight into the output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads the key behavior with 'Dry-run / validate' and avoids any unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description is fairly complete. It explains the core behavior and output type, though it doesn't specify the exact structure of the feedback. Given no output schema, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of the parameter with a description. The tool description adds the context of 'CFX Python' and the validation purpose, but the parameter semantics are largely defined by the schema already.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool is for dry-run/validation of CFX Python and mentions it avoids side effects, which clearly distinguishes it from sibling run_code. It uses specific verbs and provides the resource and purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use when you want to validate code without executing it, contrasted with run_code. It does not explicitly name alternatives or when-not-to-use, but the 'without applying side effects' gives clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_codeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full transparency burden. It discloses the execution namespace, the list of refreshed helper variables, and the return value (stdout, stderr, __return__). It also hints at which operations are not appropriate, but does not detail side effects or safety implications of arbitrary code execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each with purpose: first states the core action, second details helpers and return, third gives alternative guidance. No wasted words, information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of code execution and no output schema, the description covers the essential behavior, execution context, return values, and offers routing guidance. It stops short of describing potential side effects or the exact signatures of helpers, but the core is sufficiently complete for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the 'code' parameter is described, so baseline is 3. The description adds meaning beyond the schema by explaining the execution environment (helpers refreshed from CFX sessions) and the return format. However, the schema mentions 'validation, grounding, or execution' while the description only says 'execute,' creating slight ambiguity about whether validation/grounding modes exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Execute Python code against the active PyCFX session namespace.' This clearly distinguishes it from siblings like cfx_workflow and cfx_model_context by explicitly naming them as preferred for routed actions and read-only queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for use (executing code in a session) and explicitly directs users to prefer cfx_workflow or cfx_model_context for routed actions and read-only queries. This gives effective alternative guidance, though it could be more explicit about exactly when run_code is the best choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly discloses that the call is safe (no side effects) and describes the return contents, which covers the key behavioral aspects for a read-only status check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, and every word earns its place. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params) and presence of an output schema, the description is largely complete. It adds the important context of safety before connect and the list of available tools. Minor omission is lack of detail on the case when no backend is active, but the output schema likely covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to document. The baseline for 0 params is 4, and the description correctly ignores parameters entirely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Report whether the cfx leaf has an active backend.' It also lists what it returns (endpoint, backend kind, tools), making it clearly distinct from siblings like connect and disconnect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Safe to call before `connect`', providing a clear when-to-use context. It doesn't mention alternatives or exclusions, but the use case is unambiguous for a status-reporting tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.