Server Quality Checklist

Tool Scores

• beagle_get_test_resultC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, it doesn't specify authentication requirements, rate limits, error conditions, or what constitutes 'detailed' results. The JSON format mention is helpful but insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just 7 words, front-loading the core purpose. However, it may be too brief given the lack of annotations and presence of similar sibling tools, potentially sacrificing necessary context for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and multiple similar sibling tools, the description is insufficient. It doesn't explain what 'detailed test results' include, how they differ from test status, or provide any context about the testing system. The JSON format mention is the only helpful addition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents both parameters adequately. The description adds no additional parameter context beyond what's in the schema, so it meets the baseline expectation but doesn't provide extra value like explaining token relationships or format requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool 'Get detailed test results in JSON format', which clearly indicates a retrieval operation with a specific output format. However, it doesn't distinguish this from sibling tools like 'beagle_get_test_status' or 'beagle_list_test_sessions', leaving ambiguity about when to use each test-related get operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives. With multiple test-related sibling tools (get_test_status, list_test_sessions, list_running_tests), the description offers no context about prerequisites, timing, or differentiation from these similar tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_get_domain_signatureC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Get' implies a read operation, but doesn't specify if this requires authentication, has rate limits, returns structured data, or involves side effects. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place by conveying essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what a domain verification signature is, why it's needed, or what the return value looks like. For a tool with no structured behavioral data, the description should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'applicationToken' documented as 'Application token'. The description adds no additional meaning beyond this, such as explaining what an application token is or how it relates to domain verification. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get domain verification signature' clearly states the action (get) and the resource (domain verification signature), making the purpose understandable. However, it doesn't differentiate this tool from its sibling 'beagle_verify_domain', which appears related to domain verification, leaving some ambiguity about when to use each tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'beagle_verify_domain' or other domain-related operations. It lacks context about prerequisites, such as whether domain verification must be set up first, or when this signature is needed in the workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_modify_applicationC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the tool modifies an existing application, implying mutation, but lacks details on permissions, side effects, error conditions, or response format. For a mutation tool with zero annotation coverage, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no wasted words. It's front-loaded and efficiently conveys the core action. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool modifies an application (a mutation) with no annotations and no output schema, the description is incomplete. It should explain behavioral aspects like what happens on success/failure, required permissions, or constraints. The current description leaves too many gaps for effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all three parameters documented in the schema. The description doesn't add any meaning beyond the schema, such as explaining relationships between parameters or usage examples. Baseline 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Modify an existing application' states the verb (modify) and resource (application), making the purpose clear. However, it lacks specificity about what aspects are modified and doesn't differentiate from sibling tools like 'beagle_modify_project' beyond the resource name. It's adequate but vague about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing application), exclusions, or comparisons to siblings like 'beagle_create_application' or 'beagle_delete_application'. Usage is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_modify_projectC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it modifies a project without disclosing behavioral traits. It doesn't mention whether this is a destructive operation, permission requirements, error handling, or what happens on success/failure. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations, no output schema, and 3 required parameters, the description is incomplete. It doesn't address what the tool returns, error conditions, or side effects, leaving significant gaps for an agent to use it correctly in context with siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are documented in the schema. The description adds no additional meaning about parameters beyond implying modification occurs. It doesn't explain relationships between parameters (e.g., that projectKey identifies the target, while name/description are updated fields), so it meets the baseline but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Modify an existing project' clearly states the action (modify) and resource (project), distinguishing it from creation/deletion siblings. However, it lacks specificity about what aspects can be modified (e.g., name, description) or the scope of changes, making it somewhat vague compared to more detailed alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like beagle_create_project or beagle_delete_project. It doesn't mention prerequisites (e.g., needing an existing project key) or contextual constraints, leaving usage entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_verify_domainC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. 'Complete domain verification' implies a write operation that finalizes verification, but it doesn't specify whether this is idempotent, what permissions are required, if it triggers side effects (e.g., notifications), or what happens on failure. Critical behavioral traits like error handling or rate limits are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two words, front-loading the core action without any fluff. Every word ('complete', 'domain verification') contributes directly to understanding the tool's purpose, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of domain verification (likely involving security and configuration changes), no annotations, and no output schema, the description is insufficient. It doesn't explain what 'complete' entails (e.g., does it return a status, trigger a process?), what happens after verification, or potential side effects. For a tool that presumably modifies system state, this leaves significant gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are well-documented in the schema itself. The description adds no additional meaning about parameters beyond implying they're used for verification. It doesn't explain how 'signatureType' or 'pluginType' relate to the verification process, nor does it provide examples or contextual usage for these enums.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('complete domain verification') but lacks specificity about what domain verification entails or what resource it operates on. It distinguishes from siblings like 'beagle_get_domain_signature' by focusing on completion rather than retrieval, but doesn't clearly differentiate from other verification-related tools that might exist in the broader context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a domain setup first), exclusions, or relationships to sibling tools like 'beagle_get_domain_signature' or 'beagle_start_test'. The description alone offers no context for appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_create_applicationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a creation operation, implying it's a write/mutation, but doesn't cover critical aspects like required permissions, whether the operation is idempotent, what happens on duplicate names, or typical response format. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place, and there's no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool (creation) with no annotations and no output schema, the description is incomplete. It doesn't address behavioral traits like side effects, error conditions, or return values, which are crucial for an agent to use it correctly. The high schema coverage helps with inputs, but overall context for safe and effective invocation is lacking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all four parameters (name, url, projectKey, type) with descriptions and enum values. The description adds no additional parameter semantics beyond implying these are needed for creation. This meets the baseline of 3, as the schema does the heavy lifting, but the description doesn't enhance understanding of parameter roles or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and resource ('new application in a project'), making the purpose immediately understandable. It distinguishes from siblings like 'beagle_modify_application' (update) and 'beagle_delete_application' (remove), though it doesn't explicitly contrast with them. The description is specific but could be more precise about what constitutes an 'application' in this context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing project), exclusions, or comparisons to siblings like 'beagle_modify_application' for updates. The agent must infer usage from the tool name and context alone, which is insufficient for optimal selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_create_projectC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose permissions required, whether the operation is idempotent, rate limits, or what happens on success/failure. For a creation tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a creation operation with no annotations and no output schema), the description is incomplete. It lacks information on behavioral traits, usage context, and expected outcomes, which are critical for an agent to invoke it correctly in a real-world scenario.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters (name, description, teamId). The description adds no additional parameter semantics beyond what's in the schema, such as format constraints or examples, resulting in the baseline score for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Create') and resource ('new project in Beagle Security'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'beagle_modify_project' or explain what distinguishes creating a project from creating an application ('beagle_create_application'), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., authentication needs), when not to use it, or how it relates to sibling tools like 'beagle_modify_project' or 'beagle_create_application', leaving the agent with no context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_delete_applicationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Delete' implying a destructive mutation, but doesn't specify if deletion is permanent, reversible, requires specific permissions, or has side effects (e.g., cascading deletions). This leaves critical behavioral traits undocumented for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero waste. It is front-loaded and efficiently conveys the core action without unnecessary words, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It doesn't cover behavioral aspects like permanence or permissions, nor does it explain the outcome (e.g., success confirmation or error handling). Given the complexity of deletion operations, more context is needed for safe use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the parameter 'applicationToken' clearly documented. The description adds no additional meaning beyond the schema, such as explaining what an application token is or where to find it. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('an application'), making the purpose immediately understandable. It distinguishes from siblings like 'beagle_get_application' or 'beagle_modify_application' by specifying deletion. However, it doesn't specify what type of application (e.g., Beagle application) or provide additional context about the resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing application token), exclusions, or comparisons with siblings like 'beagle_delete_project' for related operations. The description lacks context for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_delete_projectC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but provides minimal behavioral insight. 'Delete' implies a destructive, irreversible action, but it doesn't specify permissions needed, confirmation steps, side effects (e.g., cascading deletion), or error handling. This is inadequate for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just three words—with zero wasted language. It's front-loaded and to the point, though this brevity contributes to gaps in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It lacks critical context like success/error responses, safety warnings, or dependencies, leaving the agent under-informed about the tool's behavior and implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'projectKey' parameter. The description adds no additional meaning about the parameter, such as format examples or constraints beyond deletion. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('a project'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'beagle_delete_application' or 'beagle_modify_project', but it's not vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites, irreversible consequences, or when to choose deletion over modification ('beagle_modify_project') or other related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_get_applicationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states a read operation ('Get'), implying non-destructive behavior, but doesn't disclose error handling, authentication needs, rate limits, or response format. For a tool with no annotation coverage, this is a significant gap in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise and front-loaded with a single, clear sentence. Every word earns its place, with no redundancy or fluff, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'details' include, error behaviors, or response structure. For a tool that likely returns application data, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'applicationToken' documented in the schema. The description adds minimal value by implying the token is used to retrieve details, but doesn't provide additional context like token format or examples. Baseline 3 is appropriate as the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and target ('application details'), and specifies the lookup method ('by token'). It distinguishes from siblings like 'beagle_list_applications' (which lists multiple) and 'beagle_modify_application' (which updates). However, it doesn't specify what 'details' include, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid token) or contrast with siblings like 'beagle_list_applications' for browsing. The description implies usage when you have a token, but lacks context on error cases or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_get_test_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it gets status, implying a read-only operation, but doesn't cover aspects like authentication needs (implied by 'applicationToken'), rate limits, error conditions, or what the status output entails (e.g., pending, running, completed).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's appropriately sized for a simple tool and front-loads the core action ('Get the status').

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with 2 required parameters) and lack of annotations and output schema, the description is incomplete. It doesn't explain what the status output looks like, error handling, or how it fits into the broader test workflow with siblings like 'beagle_start_test'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('applicationToken' and 'resultToken'). The description doesn't add any meaning beyond what the schema provides, such as explaining how to obtain these tokens or their relationship to test status.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('status of a running test'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'beagle_get_test_result' or 'beagle_list_running_tests', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a started test), exclusions, or how it differs from similar tools like 'beagle_get_test_result' or 'beagle_list_running_tests'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_list_applicationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states the basic action without disclosing behavioral traits such as pagination, rate limits, permissions required, or what 'all applications' entails (e.g., includes deleted ones?). This leaves significant gaps for a tool that likely returns a list.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, front-loading the core action and resource. It's appropriately sized for a simple list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that likely returns a list of applications. It doesn't explain return values, error conditions, or behavioral details, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description implies the 'projectKey' parameter is needed to scope the listing, but the schema already has 100% coverage with a clear description ('Project key'). No additional semantics, like format examples or constraints, are provided beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('applications under a project'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'beagle_list_projects' or 'beagle_list_test_sessions' beyond the resource type, missing explicit sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'beagle_get_application' for single applications or 'beagle_list_projects' for listing projects. The description lacks context on prerequisites, filtering options, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_list_projectsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the action without disclosing behavioral traits. It doesn't mention permissions, rate limits, pagination, or output format, leaving critical operational details unspecified for a list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It is appropriately sized and front-loaded, clearly stating the core functionality without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a list operation that likely returns structured data, the description is incomplete. It fails to explain what information is returned, how results are formatted, or any limitations, leaving significant gaps for agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema fully documents the single parameter. The description adds no parameter-specific information beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resources ('projects and applications'), making the purpose understandable. However, it doesn't differentiate from sibling 'beagle_list_applications', which appears to be a more specific version, leaving some ambiguity about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'beagle_list_applications' or other list tools. The description lacks context about prerequisites, timing, or exclusions, offering minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_list_running_testsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states it's a list operation, implying read-only, but doesn't disclose behavioral traits like pagination, rate limits, authentication requirements, or what 'running' means (e.g., active vs. pending). The description is minimal and lacks context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise with a single, front-loaded sentence that directly states the tool's purpose. No wasted words or redundant information, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that likely returns a list of tests. It doesn't explain return values (e.g., format, fields), error conditions, or behavioral aspects like real-time updates. For a list operation with potential complexity, this leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with teamId documented as optional for team tests. The description adds marginal value by implying the parameter's purpose ('for user or team'), but doesn't provide syntax, format details, or examples beyond what the schema already states. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('running tests'), with scope ('for user or team'). It distinguishes from siblings like beagle_list_test_sessions (which lists sessions, not running tests) and beagle_get_test_status (which gets status of a specific test). However, it doesn't explicitly differentiate from beagle_get_test_result, which could be ambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It doesn't specify prerequisites (e.g., authentication needs), when to use teamId vs. user context, or how it differs from beagle_list_test_sessions. The description implies usage but lacks explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_list_test_sessionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it's a list operation. It doesn't disclose behavioral traits such as pagination, rate limits, authentication needs, or what 'all' entails (e.g., historical vs. active sessions). This leaves significant gaps for a tool with potential complexity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero waste. It is appropriately sized and front-loaded, clearly stating the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It lacks details on return values, error conditions, or behavioral constraints. For a list operation that might involve pagination or filtering, this leaves the agent with insufficient context to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('applicationToken' and 'count'). The description adds no additional meaning beyond implying filtering by application, which is covered by the required parameter. Baseline 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('test sessions for an application'), making the purpose understandable. It distinguishes from siblings like 'beagle_list_applications' or 'beagle_list_projects' by specifying test sessions, but doesn't explicitly contrast with 'beagle_list_running_tests' which might overlap in scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'beagle_list_running_tests' or 'beagle_get_test_status'. The description implies usage for listing all test sessions, but lacks context on prerequisites, timing, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_start_testC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Start' implies a write/mutation operation, it doesn't disclose what permissions are needed, whether this is idempotent, what happens if a test is already running, rate limits, or what the expected outcome is. The description is too minimal for a tool that initiates potentially resource-intensive operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence with no wasted words. It's front-loaded with the core action and resource. While it may be too minimal for completeness, it earns full marks for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that initiates automated penetration tests (a complex, potentially destructive operation) with no annotations and no output schema, the description is inadequate. It doesn't explain what 'starting a test' entails, what gets initiated, expected duration, or what the user should do next. Given the context of security testing tools, more behavioral context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'applicationToken' parameter. The description adds no additional parameter context beyond what's in the schema. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start') and the resource ('an automated penetration test'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'beagle_stop_test' or 'beagle_get_test_status' beyond the obvious action difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (like needing an existing application/project), what happens after starting a test, or when to use 'beagle_stop_test' or 'beagle_get_test_status' instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• beagle_stop_testC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Stop' implies a mutation/control action, it doesn't specify whether this is destructive (does it delete test data?), what permissions are required, or what happens after stopping (does it return status?). The description is minimal and lacks critical behavioral context for a control operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just three words, with zero wasted language. It's front-loaded with the core action and target, making it immediately understandable despite its brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation/control tool with no annotations and no output schema, the description is inadequate. It doesn't explain what 'stopping' entails behaviorally, what the expected outcome is, or how this interacts with other tools in the system (particularly 'beagle_start_test' and 'beagle_list_running_tests'). The minimal description leaves too many questions unanswered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter 'applicationToken'. The description adds no additional parameter information beyond what's in the schema, meeting the baseline for high schema coverage but not providing extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Stop') and the target ('a running test'), providing a specific verb+resource combination. However, it doesn't differentiate this tool from its sibling 'beagle_start_test' beyond the obvious verb difference, missing an opportunity to clarify their complementary relationship.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (like needing a test to be running), nor does it reference the sibling 'beagle_list_running_tests' which would logically be used first to identify tests to stop.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.