Server Quality Checklist

Tool Scores

• execute_codeA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explains result handling, real-time stdout/stderr via resource subscriptions, and return values. However, it does not disclose potential side effects or destructive actions (e.g., modifying scene state) beyond the general notion of code execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear title, Args/Returns sections, a Note, and an Example. It is front-loaded with the purpose, and every sentence serves a purpose without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no output schema, and no annotations, the description covers parameter semantics, return values, and side effects (real-time output). It lacks error handling or session existence checks, but is largely complete for the intended use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must provide full meaning. It does so by detailing each parameter: 'code' (Python code), 'result_type' (with three modes explained), and 'session_key' (optional). This adds substantial value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes Python code in a Maya session. This verb-resource pair is distinct from sibling tools (list_sessions, write_module, add_session) which do not involve code execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use (for executing code in Maya) with parameter explanations and examples. However, it does not explicitly exclude alternatives or state when not to use, leaving room for clearer differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_sessionA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes the action as manual addition, notes prerequisites (open command port), and mentions return type. Lacks detail on potential side effects, but given no annotations, it is fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (use case, args, returns, prerequisites). The code example adds length but is helpful. Concise overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers when to use, prerequisites, parameter defaults, and return type. Adequate for a two-parameter tool with an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameters are described minimally as host and port, with defaults. Schema coverage is 0%, so description adds some value but doesn't provide deeper semantics like valid ranges or formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states it manually adds a Maya session at a specific host and port, distinguishing from auto-discovery. Clear verb-resource combination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use when auto-discovery fails or for specific ports, and provides prerequisite steps with code example.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_moduleA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It transparently explains key behaviors: automatic creation of parent packages for dotted names, overwrite behavior (replace or error), and optional session key. It does not cover potential side effects on the Maya session or persistence, but what is disclosed is accurate and helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with an Args, Returns, and Example section, but it is somewhat lengthy. It includes a multi-line example that takes space. The core purpose is front-loaded, but conciseness could be improved by trimming the example or combining sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (though not shown, context signals confirm), the description complements it well. It explains the return as a 'Success message' and covers parameters comprehensively. It lacks details on error conditions or scope of the virtual module, but overall it provides a complete picture for a tool of moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate, and it does excellently. Each parameter is explained in the Args section: 'name' can be a dotted path with auto-creation of parents, 'code' is Python source, 'overwrite' controls replacement behavior, 'session_key' is optional. The example demonstrates real usage, adding significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create a virtual Python module in a Maya session.' It uses a specific verb ('Create') and identifies the resource ('virtual Python module'), making it easy to understand. This purpose is distinct from sibling tools like list_sessions, execute_code, and add_session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an example that illustrates typical usage and implies that write_module is for creating reusable modules, while execute_code runs code directly. However, it does not explicitly state when to use this tool over alternatives or when not to use it, which would strengthen guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sessionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes that it returns session information and mentions background scanning. No annotations provided, so description carries burden. Could be more transparent about data freshness or performance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with clear first sentence, bullet-like list of fields, and usage note. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters, output schema available, and low complexity, the description fully covers purpose, return data, and usage hint without gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; baseline is 4 per guidelines. Description adds no param info, which is acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'List all active Maya sessions' and enumerates returned fields. Distinguishes from sibling tools (write_module, execute_code, add_session) which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes note about periodic polling for change detection, providing context. Lacks explicit alternatives or when-not-to-use, but usage is implied by the sibling set.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.