Server Quality Checklist

Tool Scores

• get_project_statusD
  1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds no behavioral information beyond what annotations already provide (readOnlyHint, idempotentHint, etc.). It does not disclose any additional traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence that is under-specified, wasting the opportunity to convey useful information. It is not helpfully concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fails to explain what the tool returns, how to interpret status, or any contextual details. With no output schema and minimal description, the agent cannot use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain the projectKey parameter or its purpose. No added meaning beyond schema constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description merely restates the tool name and says 'use this tool'. It does not specify the action (getting status of a project) or what resource is involved, making it tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings like list_projects or start_workspace_scan. No context or prerequisites provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_fix_taskD
  1.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the tool is safe. However, the description adds no behavioral context beyond what annotations provide, such as what data is returned or any prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise but at the expense of clarity. The single sentence is not informative and reads as under-specification rather than efficient communication.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no output schema, and several sibling tools (list_findings, prepare_workspace_scan, etc.), the description is wholly inadequate. It leaves the agent without a clear understanding of when or how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the schema itself provides no parameter explanations. The description does not clarify that scanId is a UUID for the scan, findingId identifies the finding, or that locale is optional for language. No value added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description is tautological: 'Use the authenticated LaunchLint cloud tool get_fix_task' merely repeats the tool name without specifying that it retrieves a fix task for a given finding. It fails to distinguish from siblings like list_findings or get_project_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. For instance, it does not indicate that it should be called after listing findings or that it provides details for a specific finding.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_findingsD
  1.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, openWorldHint, and destructiveHint=false, handling safety and idempotency. However, the description adds no behavioral context beyond stating 'authenticated', which is implied. No additional traits (e.g., pagination, filtering behavior) are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence but is a tautology that wastes space. It is brief but uninformative, not earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and three parameters, the description should explain return values or behavior. It provides none, making it wholly incomplete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain parameters but fails to do so. It does not mention scanId, domain, or severity, leaving their meaning and usage entirely undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is a tautology: it simply repeats the tool name and adds 'authenticated LaunchLint cloud tool', failing to state what the tool does (e.g., 'list findings from a scan'). No verb or resource is specified beyond the name itself, making it misleading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. Sibling tools like list_projects and get_project_status exist, but the description provides no differentiation or context for appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsC
  2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, but the description adds no behavioral insight beyond that. It does not contradict annotations, but contributes no additional transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short (one sentence) but fails to convey the tool's purpose. It is under-specified rather than concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no parameters and no output schema, the description should clearly state what it does (e.g., 'Lists all projects'). It fails to provide even this basic context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so the description does not need to explain parameter semantics. Baseline score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is a tautology: it restates the tool name without explaining what 'list_projects' does. It provides no verb or resource beyond the name itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus siblings like get_project_status or list_findings. The phrase 'authenticated LaunchLint cloud tool' implies context but is insufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_workspace_scanA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations lack destructiveHint and idempotentHint detail, but the description adds meaningful behavioral context: it consumes a resource ('app check') and requires explicit confirmation (confirm=true). This goes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, 16 words, with the verb 'Upload' first. Every word adds value; no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, and the description omits return value (e.g., scan ID, status). It also fails to mention that the snapshot must be prepared via sibling 'prepare_workspace_scan', leaving a gap in workflow understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description provides no explanation of the 5 parameters. The existence of 'confirmationToken' and 'confirm' is only hinted at by 'requires explicit confirmation', leaving the agent to infer parameter usage from schema constraints alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (upload snapshot), the target system (LaunchLint), and key constraints (requires confirmation, consumes one app check). It distinguishes this from sibling 'prepare_workspace_scan' by implying it follows preparation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used after preparing a snapshot but does not explicitly state when to use versus alternatives, nor does it provide exclusion criteria or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_workspace_scanA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds value by specifying the tool does not upload and returns specific data including a one-time token, which is not obvious from annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loading the core action (inspect without upload) and listing return values. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only inspection tool with one optional parameter and no output schema, the description covers the key points: what it does, what it returns, and a token note. It could mention the token's purpose (e.g., for later confirmation) but is otherwise adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the platforms parameter, but it does not. The parameter is optional with a simple enum, but the agent cannot infer that platforms filters the inspection scope.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool inspects the approved workspace without uploading it, and lists what it returns (counts, exclusions, size, confirmation token). This differentiates it from siblings like start_workspace_scan, which likely uploads or initiates a scan.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context (inspect without upload, returns token) implying use as a preparatory step before scanning. It does not explicitly state exclusions or alternatives, but the context is sufficient for an agent to infer when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.