Server Quality Checklist

Tool Scores

• clear_logsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. 'Clear' implies destructive action, but it doesn't specify whether logs are permanently deleted, recoverable, or archived. No information about permissions, side effects, or system impact is included.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (4 words) and front-loaded with the core action. Every word earns its place with zero wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is insufficient. It doesn't explain what 'clear' entails operationally, what data is lost, whether there are confirmation prompts beyond the parameter, or what happens after execution.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'confirm' parameter thoroughly. The description adds no additional parameter context beyond what's in the schema, maintaining the baseline score for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('clear') and target resources ('log buffer and files'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'get_logs' or explain how this differs from other maintenance operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives. The description doesn't mention prerequisites, appropriate contexts, or warnings about when NOT to use it (e.g., during debugging).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_logsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Retrieve log entries' implies a read-only operation but doesn't specify whether this requires authentication, has rate limits, returns paginated results, or what format/logic governs the retrieval. For a log tool with zero annotation coverage, this leaves significant behavioral questions unanswered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient phrase that gets straight to the point with zero wasted words. It's appropriately sized for a simple retrieval tool and front-loads the core purpose immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters, no annotations, and no output schema, the description is inadequate. 'Retrieve log entries' doesn't explain what gets returned (format, structure, limitations), doesn't address authentication or access requirements, and provides no context about the log system being accessed. For a tool with this complexity and zero structured metadata, the description should do more.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are well-documented in the schema itself. The description adds no additional parameter information beyond what's already in the structured schema. This meets the baseline expectation when schema does the heavy lifting, but doesn't provide extra semantic context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Retrieve log entries' clearly states the verb (retrieve) and resource (log entries), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'clear_logs' or 'get_tomcat_status', which would require more specific scope information to earn a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'clear_logs' (destructive) and 'get_tomcat_status' (status monitoring), there's no indication of when this retrieval tool is appropriate versus those other options, nor any prerequisites or constraints mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restart_tomcatC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Stop and start') but lacks details on permissions required, side effects (e.g., downtime, service interruption), error handling, or response format. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence ('Stop and start Tomcat') with zero wasted words. It's front-loaded and appropriately sized for the tool's complexity, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with potential side effects), no annotations, and no output schema, the description is incomplete. It fails to address critical aspects like behavioral traits, usage context, or output expectations, leaving significant gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the input schema fully documents both parameters ('force' and 'gradle_command'). The description adds no parameter-specific information beyond what's in the schema, resulting in a baseline score of 3 where the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Stop and start') and target resource ('Tomcat'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'start_tomcat' or 'stop_tomcat', which prevents a score of 5, but it's specific enough to avoid vagueness or tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like the sibling tools 'start_tomcat' or 'stop_tomcat'. There's no mention of prerequisites, use cases, or exclusions, leaving the agent to infer usage from context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_tomcatC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action ('Launch') but doesn't describe what happens during execution (e.g., whether it blocks until Tomcat is ready, if it runs in background, error handling, or side effects like port binding). This leaves significant gaps for a tool that likely involves system processes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and method, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that likely involves complex system behavior. It doesn't explain what 'Launch' entails operationally (e.g., success criteria, timeouts, or output format), leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (gradle_command and working_directory). The description adds no additional parameter semantics beyond what's in the schema, but the baseline is 3 since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Launch') and target ('Tomcat') with the method ('via Gradle'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from siblings like 'restart_tomcat' or 'get_tomcat_status' in terms of initial startup versus other lifecycle operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'restart_tomcat' or 'stop_tomcat'. It lacks context about prerequisites (e.g., Tomcat must be stopped first) or typical scenarios for launching versus restarting, leaving the agent to infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stop_tomcatC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Terminate' implies a destructive action, but it doesn't specify permissions required, whether termination is graceful or immediate by default, or what happens post-termination (e.g., error handling). The description adds minimal value beyond the obvious implication of stopping a process.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place in conveying the essential purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a destructive operation with no annotations and no output schema, the description is incomplete. It doesn't address critical context like what 'terminate' entails (e.g., SIGTERM vs. SIGKILL), error scenarios, or return values, leaving significant gaps for an agent to handle this tool safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'force' parameter clearly documented in the schema. The description adds no additional parameter information beyond what the schema provides, so it meets the baseline score of 3 for adequate but unenhanced parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Terminate Tomcat process' clearly states the action (terminate) and target resource (Tomcat process), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'restart_tomcat' or 'get_tomcat_status' beyond the basic verb difference, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'restart_tomcat' or 'get_tomcat_status'. It lacks context about prerequisites (e.g., Tomcat must be running), consequences (e.g., service interruption), or typical use cases, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tomcat_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Check Tomcat process status' implies a read-only, non-destructive operation, but it doesn't specify what 'status' includes (e.g., running/stopped, memory usage, uptime), potential errors (e.g., if Tomcat isn't installed), or side effects (e.g., minimal performance impact). For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description 'Check Tomcat process status' is a single, clear sentence with no wasted words. It's front-loaded with the core action and resource, making it immediately understandable. Every word earns its place, and there's no unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimal but incomplete. It lacks details on what the status check entails, possible return values (e.g., a status message or error), and how it fits into the sibling toolset (e.g., used before 'restart_tomcat'). For a tool with no structured output, the description should provide more context about the expected behavior and results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics beyond what the schema provides. A baseline score of 4 is appropriate as the description doesn't introduce confusion or redundancy regarding parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Check Tomcat process status' clearly states the verb ('Check') and resource ('Tomcat process status'), making the purpose immediately understandable. It distinguishes from siblings like 'get_logs' (which retrieves log files) and 'restart_tomcat' (which performs an action), though it doesn't explicitly differentiate from them. The description is specific but lacks explicit sibling comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., Tomcat must be installed), when it's appropriate (e.g., for monitoring vs. troubleshooting), or contrast with siblings like 'get_logs' for deeper diagnostics. Usage is implied by the name but not articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.