Server Quality Checklist

Tool Scores

• get_class_referenceB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states the operation (get public members) but does not reveal whether the tool is read-only, if it requires specific permissions, how it handles errors (e.g., invalid class name), or any rate limits or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two lines, front-loading the core purpose. The separate 'Args' block is clear and minimal. Every sentence adds value, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, output schema present), the description covers the essential information. It does not, however, mention error handling or assumptions about class existence. This is a minor gap but overall sufficient for an experienced developer or agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% for the single parameter class_name, so the description must compensate. It provides a brief description and examples (e.g., AActor, ACharacter), which add value beyond the raw schema. However, it does not explain naming conventions, case sensitivity, or the effect of invalid inputs. This is adequate but not comprehensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'all public members of an Unreal Engine class,' making the tool's purpose immediately obvious. The provided examples (e.g., AActor) and the distinction from sibling tools (deprecation, function signature, etc.) further reinforce uniqueness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives like search_unreal_api or get_function_signature. It does not provide context, exclusions, or prerequisites. The agent is left to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_include_pathA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the function but does not disclose edge cases (e.g., what happens if the name is invalid, case sensitivity, or whether multiple results exist). Minimal behavioral context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and to the point, with the main action front-loaded. It could be slightly more structured by integrating the parameter explanation into the first sentence, but overall it is efficient and avoids unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple tool with one parameter and an output schema (which covers return format). However, it lacks information about error handling or validation, leaving some gaps about tool behavior in edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description provides examples (e.g., 'AActor', 'FHitResult') and clarifies the parameter expects a class/struct/type name, adding significant meaning beyond the schema's bare type definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool retrieves the #include path for Unreal Engine classes/structs/types. The verb 'Get' and resource 'include path' are specific, and it is distinct from siblings like 'get_class_reference' or 'search_unreal_api'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage from sibling list but no explicit guidance on when to use this over alternatives. Lacks any 'when not to use' or prerequisite information, though the purpose is straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_deprecation_warningsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral aspects. It only states it checks deprecation, with no mention of side effects, rate limits, or that it is a read-only operation. The output schema exists but the description adds no behavioral context beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, containing one sentence for the purpose and a brief parameter doc. No unnecessary words, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has a single parameter and an output schema, the description is adequate for a simple check. However, it lacks an example or note about return format, though the output schema compensates. It fits well with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, but the description provides clear semantics for the single parameter 'name', explaining it can be a function, class, property, etc. This adds meaningful guidance beyond the raw schema definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks deprecation status of an Unreal Engine API, using the verb 'check' and specifying the resource. It distinguishes from sibling tools like 'get_class_reference' and 'get_function_signature' which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit when-to-use or when-not-to-use guidance. While the intent is clear from the name and context, there is no mention of alternatives or prerequisites, relying on the implied context of sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.