Server Quality Checklist

Tool Scores

• get_propertiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'gets' properties, implying a read-only operation, but doesn't specify permissions, rate limits, error handling, or what 'standard properties' entail. This leaves significant gaps in understanding the tool's behavior beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool with two parameters and potential complexity in Roblox instance handling. It doesn't explain return values, error conditions, or behavioral nuances, which are critical for an agent to use the tool effectively in this context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents both parameters ('path' and 'properties') with clear descriptions. The description adds no additional meaning beyond what the schema provides, such as examples or edge cases, but doesn't need to compensate for gaps, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and target ('properties of a Roblox instance'), which is specific enough to understand the tool's function. However, it doesn't differentiate from sibling tools like 'read_script' or 'list_children', which might also involve reading instance data, so it doesn't achieve full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_children' or 'read_script'. It lacks context about scenarios where fetching properties is preferred over other operations, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_childrenC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool lists children but doesn't describe what 'children' means in this context (e.g., nested instances), whether it's read-only, if there are rate limits, or what the output format looks like. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is appropriately sized and front-loaded, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of listing children in a Roblox instance, no annotations, and no output schema, the description is incomplete. It doesn't explain what 'children' entails, the return format, or any behavioral traits like safety or performance considerations, which are crucial for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'path' parameter documented as 'The path to the instance to list children of (e.g., 'game.Workspace')'. The description adds no additional meaning beyond this, so it meets the baseline of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List children') and target resource ('of a Roblox instance'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'get_properties' or 'read_script', which also involve Roblox instances but perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_properties' (which might retrieve properties rather than children) or other sibling tools. It lacks explicit context, prerequisites, or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_scriptC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool executes code, implying it's a mutation/write operation, but doesn't disclose critical traits like security implications, execution environment constraints, error handling, or potential side effects. This is a significant gap for a tool that runs arbitrary code.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero wasted words. It's front-loaded with the core purpose and efficiently conveys the essential information without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing arbitrary code and the lack of annotations and output schema, the description is incomplete. It doesn't address safety, permissions, return values, or error conditions, which are crucial for an agent to use this tool effectively in a real-world scenario.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'code' clearly documented. The description adds no additional parameter semantics beyond what the schema provides, such as code format examples or execution limits. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute') and resource ('arbitrary Lua code in the Roblox client'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'read_script' or 'list_children', which are likely read-only operations, whereas this one executes code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing specific permissions), appropriate contexts, or warnings about misuse, leaving the agent without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_scriptA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses important behavioral traits: decompilation capability and the client/server execution context limitation. However, it doesn't mention permissions needed, rate limits, error conditions, or what happens with invalid paths. The transparency is adequate but incomplete for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences. The first states the core purpose, the second provides crucial exclusion criteria. Every word earns its place, and the most important information (what the tool does) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no annotations and no output schema, the description provides adequate but minimal context. It covers the core functionality and a key limitation, but doesn't address return format, error handling, or authentication requirements. Given the simplicity of the tool, this is minimally viable but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'path' parameter. The description doesn't add any additional parameter semantics beyond what the schema provides, such as path format examples or constraints. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'read' and resource 'source code of a script', with additional clarification about decompilation when needed. It distinguishes from sibling tools like 'run_script' by focusing on reading rather than execution, though it doesn't explicitly contrast with 'get_properties' or 'list_children'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when NOT to use this tool: 'Cannot read server-side scripts from a client execution context as their source is not replicated.' This gives important exclusion criteria. However, it doesn't explicitly mention when to choose this tool over alternatives like 'get_properties' for script metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.