Server Quality Checklist

Tool Scores

• depot_get_projectA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable security context beyond annotations: trust-relationship tokens carry project-token permissions and cannot reach the Depot CI API or Depot API, only container builds and the registry. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a front-loaded purpose sentence, followed by use cases and a security note. Each paragraph adds distinct information, though it is slightly longer than a purely minimal definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only single-resource tool with an output schema, the description covers purpose, use cases, and authentication semantics. Nothing needed to select and invoke the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the single projectId parameter is already documented as coming from depot_list_projects. The description adds no additional parameter-level meaning, so it does not need to compensate; baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Show one Depot container build project's full configuration together with its OIDC trust policies.' This clearly distinguishes it from sibling list/diagnostic tools by narrowing scope to a single project and its trust policies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: 'check build capacity and cache retention for a specific project' and 'audit which external CI systems are allowed to exchange an OIDC token for Depot credentials.' It does not explicitly name when-not-to-use alternatives, but the 'specific project' framing makes the boundary clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_list_ci_secretsA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare read-only, idempotent, non-destructive behavior. The description adds valuable context: secret values are never exposed by design, filtering happens server-side because upstream filters are undocumented, and the beta API is likely to change. These are meaningful behavioral disclosures beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and each paragraph earns its place: purpose and security first, then debugging guidance, then filtering and API risk. It is longer than one or two sentences, but nothing is filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only listing tool with a diverse set of optional filters, the description covers the security model, scoping semantics, debugging workflow, server-side filtering rationale, and API stability. An agent has enough context to invoke the tool correctly and interpret its results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents each parameter. The description adds conceptual value by explaining how scoping works — a secret has variants scoped by repository, environment, branch, and workflow — which helps the agent understand why filters matter and how to interpret results. This goes beyond the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists Depot CI secret names and scoping, and explicitly notes that values are never returned. It is specific about the resource and action, but it does not explicitly distinguish itself from sibling tools like depot_list_ci_variables or depot_diagnose_ci_failure, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit use case: answer 'why can't my job see $FOO' and compare returned scoping against the job being debugged. It does not mention alternatives or provide when-not-to-use guidance, so it is not a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_list_imagesA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint/idempotentHint/destructiveHint annotations, the description explicitly states the operation cannot delete tags or images and explains WHY deletion is not exposed—it is irreversible and something may be deploying what you delete. It also discloses the DEPOT_PROJECT_ID fallback behavior, adding context the annotation booleans alone do not convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence carries the core purpose, followed by short paragraphs for use cases, environment/parameter behavior, and a safety note. Every sentence earns its place, though four paragraphs is slightly more than strictly necessary for a simple list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only list tool with an output schema, 100% schema parameter coverage, and four safety annotations, the description covers purpose, use cases, the projectId mechanism, and the read-only guarantee. Nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all three parameters are already fully documented (limit bounds and default, pageToken semantics, projectId fallback). The description only restates the projectId fallback in prose without adding new parameter-level detail, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'List the container images stored in a Depot project's registry, with tag, digest, push time, and size.' It names the exact returned fields, and no sibling tool covers image listing, so it is trivially distinguishable from the other fourteen depot_* tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Three concrete use cases are given: verifying a build pushed what was expected, finding the digest behind a tag before deploy, and checking when an image was last refreshed. It also routes the agent to depot_list_projects when the projectId is unknown. No alternative tool is named for exclusion, but the provided scenarios make selection unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_list_ci_artifactsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds important behavioral detail beyond those annotations: signed URLs are minted one request per artifact, capped at 10 per call, and the tool only returns metadata and links without downloading contents. This is exactly the kind of context an agent needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: the first sentence captures the core operation, the second gives practical use cases, and the third explains the optional signed-URL behavior and its constraints. No sentence is wasted, and the most decision-relevant detail (never downloads contents) is included early.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present and annotations covering safety, the description provides the remaining behavioral context an agent needs: what artifacts are for, how to narrow results, how signed URLs behave, and the cap. Pagination is handled by the schema's pageToken description, so nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents every parameter. The description adds some relational context (e.g., passing jobId/attemptId alongside runId, withDownloadUrl costing extra requests) but does not substantially expand beyond the schema's own descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List'), a precise resource ('artifacts'), and the exact scope ('a Depot CI run, workflow, job, or attempt'). It also clarifies what the tool does not do ('never downloads or reads artifact contents'), making its purpose unmistakable even among many sibling CI tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: 'Use this to find out what a run produced' with concrete examples, and explains when to filter by job or attempt. It does not explicitly compare against sibling tools like depot_get_ci_logs or depot_get_ci_job_summary, so an agent is left to infer those boundaries, but the guidance is otherwise strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_list_projectsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description does not need to repeat safety traits. It adds useful context beyond annotations by specifying what fields are returned and by clarifying the tool's scope relative to Depot CI runs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and payload, then provides concrete use cases and an explicit sibling distinction. Every sentence earns its place and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of this read-only list tool, the presence of an output schema, and strong annotations, the description is complete. It tells the agent what to use the tool for, what it returns, and when to choose a different tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all three parameters with 100% coverage, including defaults and examples. The description adds no parameter-specific detail, but the baseline of 3 is appropriate because the schema carries the full semantic burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List'), a specific resource ('Depot container build projects'), and key returned data ('region, runner hardware, cache policy'). It clearly distinguishes itself from CI-run tools by explicitly saying these are container build projects, not CI runs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use this tool: to find a projectId for list/diagnose operations and to check build-speed-related configuration. It also explicitly tells the agent to use depot_list_ci_runs for CI runs instead, providing a clear alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_get_ci_job_summaryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool readOnly, idempotent, and non-destructive; the description adds meaningful behavior beyond that: the summary is job-authored markdown and 'an empty result is normal rather than an error.' This prevents misinterpreting empty output as failure and clarifies the accepted id forms (attempt, job, or run id). No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every paragraph earns its place: purpose, interpretation/value, empty-result handling with sibling routing, and id flexibility. The first sentence is front-loaded with the exact operation, and there is no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 100% schema coverage, an output schema, and strong annotations, the description covers purpose, usage context, empty behavior, alternatives, and id semantics. An agent has enough information to call the tool correctly and interpret the result without further inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100%, including an enum for targetType and a clear description for id. The description mostly restates the id flexibility without adding syntax or format details beyond the schema. Baseline 3 is appropriate because the schema carries the parameter-semantics weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence names a specific verb ('Read'), a concrete resource ('step summary a Depot CI job authored for itself'), and anchors it to a familiar GitHub Actions concept ($GITHUB_STEP_SUMMARY). This clearly differentiates the tool from log- or run-level tools like depot_get_ci_logs or depot_get_ci_run even before reading the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent that an empty result is normal and should not be treated as an error. It then routes to siblings: use depot_diagnose_ci_failure for a failure and depot_get_ci_logs for raw output, which is clear when-to-use vs alternatives guidance. It also explains why the summary is preferable to logs when present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_get_ci_metricsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark it read-only and idempotent, and the description adds meaningful behavioral context: metrics for a running attempt change between calls, a snapshot time is reported, and the response includes the raw document because Depot publishes no field names. It also discloses the resource-exhausted failure mode and a workaround.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then flows from usage triggers to parameter behavior to response shape and failure handling. Every sentence adds information; no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description does not need to enumerate return fields, and it still covers param semantics, changing-data behavior, response composition, and error handling. An agent has enough context to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds important semantic detail: the level is inferred when omitted, and the server tries run, then job, then attempt. This goes beyond the schema's parameter descriptions and clarifies how 'level' and 'id' interact.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Read CPU and memory metrics for a Depot CI run, job, or job attempt.' The stated use cases (OOM kills, exit 137, under-provisioned runner) set it apart from sibling tools like depot_get_ci_logs and depot_get_ci_job_summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs when to use the tool: when a job died without a useful error, was killed, hit exit 137, or is slow. It gives diagnostic patterns (memory at limit, CPU pinned) but does not name specific sibling alternatives or say when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_get_ci_runA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as readOnly, idempotent, openWorld=false, and non-destructive. The description adds important behavioral context beyond annotations: the tool only reports structure and status, does not explain failures, and returns node ids needed for drilling into logs/metrics. This is exactly the kind of limitation disclosure that helps an agent avoid misusing the result.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight paragraphs, each earning its place: first defines the output, second gives concrete usage and the flag, third states the limitation and routes to the correct alternative. No filler or repetition of annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tree tool with an output schema and full parameter schema coverage, the description supplies all required context: what the output looks like, which jobs/attempts to inspect, how to filter, what the tool does not do, and which sibling to call for root cause. Nothing an agent needs to select or invoke the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both runId and failedOnly clearly. The description reinforces failedOnly's purpose ('cut a large matrix down to just the broken jobs') but does not add meaning beyond what the schema provides. A baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource: 'Show one Depot CI run as its workflow -> job -> attempt tree' with node statuses and drill-in IDs. It distinguishes itself from siblings by explicitly noting this tool does not explain failures and that root cause belongs to depot_diagnose_ci_failure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use the tool ('Use this to see the shape of a run'), how to use failedOnly for large matrices, and when not to use it ('does not explain failures') with a clear alternative call: depot_diagnose_ci_failure. It also tells the agent which attempt ids to pass to depot_get_ci_logs or depot_get_ci_metrics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_list_buildsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only, idempotent, and non-destructive. The description adds useful behavioral context: it mentions the returned fields (cachedSteps, totalSteps, secondsSaved), the DEPOT_PROJECT_ID fallback, and that results are 'recent', which conveys ordering without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short paragraphs, each earning its place: the first defines the core function, the second gives concrete diagnostic use cases, and the third covers required context and sibling differentiation. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with an output schema, the description covers what the tool returns, how to select a project, when to use it, and what it is not. An agent has everything needed to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all three parameters with descriptions, so the baseline is 3. The description adds extra semantic value by stating 'Requires a projectId' despite the schema having no required fields, explaining the DEPOT_PROJECT_ID fallback, and pointing to depot_list_projects as a source for valid IDs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List recent container builds for a Depot project, with duration and cache effectiveness for each.' It clearly distinguishes container builds from CI runs, and the semantics are aligned with the title and annotations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'to find a build to diagnose, or to answer are our builds getting slower.' It also names the alternative: 'use depot_list_ci_runs for those' and points to depot_list_projects for project ID discovery.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_list_ci_runsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description only needs to add context beyond safety. It adds newest-first ordering, the limited return payload ('identity, status and timing only'), and the explicit absence of logs or failure detail. Pagination behavior is not described in prose, but the schema's pageToken parameter covers that gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and ordering, then moves to use cases, a recommended call pattern, and finally the boundary of what the tool returns. Every sentence earns its place and no sentence merely repeats the schema or annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only list tool with a rich input schema and an output schema, the description covers ordering, filtering, return scope, exclusions, and next steps with the returned runId. An agent can both select this tool over siblings and invoke it correctly without needing further inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by grouping the optional filters into a readable list and by providing a concrete recommended argument combination: status=['failed'] with limit=1. It does not redefine each parameter, but the invocation pattern gives the agent useful guidance beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'List recent Depot CI runs' including ordering ('newest first'). It clearly names the filter dimensions and explicitly contrasts itself with depot_diagnose_ci_failure and depot_get_ci_run by stating what it returns and what it does not. This cleanly separates it from sibling list/detail/diagnosis tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives concrete use cases ('find the run someone is talking about', 'did main go green', 'what ran for PR 412') and a recommended fast path: status=['failed'] with limit=1, then depot_diagnose_ci_failure. It also explicitly says the tool does not return logs or failure detail and directs the agent to depot_diagnose_ci_failure for that, so when-to-use and when-not-to-use are both clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_list_ci_variablesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the operation as read-only, idempotent, and non-destructive, so the description focuses on valuable extra behavior. It explains the variable-as-variants scoping model, reveals that values can be redacted when they resemble secrets, and warns that the underlying API is beta and likely to change.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then moves through use cases, behavioral nuances, and API stability in a logical order. Every sentence earns its place, and the length is appropriate for the tool's conceptual complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With annotations covering safety and an output schema present, the description only needs to explain selection and behavioral nuance, which it does thoroughly. It covers scoping, redaction, filtering location, and beta instability, leaving no important gap for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all five optional parameters with 100% coverage, so the baseline is 3. The description adds meaningful semantic context by explaining that a single variable name can have multiple variants scoped by repository, environment, branch, and workflow, and by clarifying that filtering is performed server-side.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: it lists Depot CI variables, their values, and their scoping. It also distinguishes itself from the sibling depot_list_ci_secrets by explicitly noting that variables return values while secrets do not.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly identifies the use case: answering "why can't my job see $FOO" and checking whether a variable holds the expected value for a branch or environment. It references the sibling secrets tool, but does not give a strict when-not-to-use rule beyond the contrast that variables return values.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_whoamiA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the readOnly/idempotent annotations by disclosing that it reports visible organizations/projects, always indicates no mutating capabilities, and never returns the token itself. This gives the agent important security and expectation-setting context not present in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and then adds usage guidance and security notes. It is slightly longer than strictly necessary, but each paragraph covers a distinct and valuable aspect, so the verbosity is justified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter diagnostic tool with an output schema and strong annotations, the description is complete. It covers when to call it, what it reports, a common failure scenario, and the crucial guarantee that it never exposes the token. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the baseline is 4 and parameter documentation is unnecessary. The description appropriately focuses on what the tool returns rather than inputs, which is all that is needed here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Verify the configured Depot token and report which organizations and projects it can actually see.' This clearly distinguishes it from the list/get/diagnose sibling tools and makes its diagnostic role obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs the agent to call this tool first 'whenever another Depot tool returns an empty list or a permission error,' and explains the underlying failure mode. This is concrete, actionable guidance that an agent can apply without inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_diagnose_ci_failureA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint and idempotentHint, and the description reinforces this with "Read-only: this never retries, cancels, or triggers anything." It also discloses that diagnoses are AI-generated and can be wrong, and explains the server-side bounding behavior that limits context cost.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured into clearly labeled sections: purpose, when to use, parameter guidance, how to read the returned state, and a caution about AI-generated content. Every paragraph earns its place and the most important guidance is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description correctly focuses on interpretation rather than raw return format. It explains all meaningful states, gives actionable guidance for each, names the sibling tool to use afterward, and warns about reliability. Nothing essential is missing for correct invocation and response interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful guidance beyond the schema by explaining that "id" accepts any identifier kind and the server resolves it, and that "targetType" is only needed to skip inference. This is valuable semantic context not present in the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: "Explain why a Depot CI run, workflow, job, or attempt failed." It clearly distinguishes this diagnostic tool from log retrieval and other CI tools by emphasizing root-cause analysis over raw logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says "Reach for this first whenever someone asks why CI failed" and names the alternative with a clear preference rule: "Prefer this over depot_get_ci_logs" and use logs only for more detail on a specific attempt. This leaves no ambiguity about when to choose this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_get_usageA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description adds valuable behavioral context: the data is aggregated, there is no per-job detail, projectId changes the response shape, and the default window is 30 days. It also explains the meaningfulness of minutesSaved vs minutesBilled, which goes beyond the structured schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized for a complex usage-reporting tool and is front-loaded with a concise summary of what it returns. Every sentence adds value: use cases, the uniqueness of the runner data, aggregation caveats, and scope behavior. There is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with an output schema, read-only annotations, and a rich input schema, this description is complete. It covers what data is returned, when to use it, how scope changes, exclusions, and default behavior. Nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by explaining that projectId returns build counts, duration, and layer cache size instead of the organization-wide breakdown, and by confirming the default look-back period. It does not deeply discuss startAt/endAt semantics, but the schema already covers those.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: it reports Depot usage for a period, enumerating build minutes, caching savings, runner minutes, storage, and sandbox minutes. It clearly differentiates this from sibling list/diagnose tools by framing it as the cost/spend reporting endpoint and noting it is the only API exposure for managed runner data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this for cost questions and gives concrete examples like 'what is burning our Depot minutes' and 'which repo dominates our runner bill'. It also clarifies that there is no per-job runner list, preventing misuse, and explains when to pass projectId to change scope.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_diagnose_buildA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral detail beyond the readOnlyHint and destructiveHint annotations: it cannot start, retry, or cancel a build; container builds cannot be triggered through Depot's API at all; and without a projectId it scans recent builds at extra request cost. These are meaningful, non-obvious traits that help an agent predict side effects and performance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average, but every sentence earns its place: purpose, usage trigger, differentiation from CI, parameter guidance, output significance, and safety guarantees. It is front-loaded with the core purpose and organized so the most actionable guidance appears early.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with an output schema, documented parameters, and read-only/idempotent annotations, the description still fills the remaining context: when to reach for it, why it exists, what operational costs to expect, and what the cache metrics mean. Nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though the schema already documents all three parameters at 100% coverage, the description adds critical semantics for projectId: the build record lacks the project id, the steps API requires it, and omitting it triggers extra scanning requests. It also clarifies what buildId is by referencing depot_list_builds and the dashboard, and ties tailLines to the log-tail behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and outcome: 'Explain why a Depot container build failed' and then details exactly what it returns (failing step, error, log tail, cache stats). It clearly distinguishes itself from CI diagnosis with 'Unlike Depot CI, container builds have no server-side AI diagnosis.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use the tool: "Use this for 'why did my docker build fail'." It also explains the alternative context, noting the difference from Depot CI, and gives concrete operational guidance about passing projectId, including the DEPOT_PROJECT_ID default and the cost of omitting it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• depot_get_ci_logsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description massively exceeds what the annotations (readOnlyHint, idempotentHint, destructiveHint) already provide: it discloses that grep is applied server-side after page fetches so it reduces received data but not read data, warns that grep walks up to DEPOT_MCP_MAX_LOG_PAGES pages and is the most expensive mode, documents the paging contract in full (tail vs forward, page-cap early stop, exact-line resumption), requires tokens to be passed back verbatim, and caps line bodies at 2000 chars with a bodyTruncated marker. This is deep behavioral disclosure with no contradiction against the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long, but every paragraph earns its place: purpose, sibling routing, filtering-cost model, id polymorphism, and the paging contract are all dense operational content rather than filler. It is front-loaded with purpose and the 'try depot_diagnose_ci_failure first' guidance, and the paging contract is given as a structured bulleted list, making it easy for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 8 parameters, a two-mode paging contract, polymorphic id handling, and an output schema; the description covers all of it, including defaults, the page cap, truncation markers, and even the cross-turn polling pattern ('following nextPageToken until it is absent yields every line exactly once'). With the output schema already present, no crucial invocation detail is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though schema description coverage is 100% and the baseline is therefore 3, the description adds essential meaning beyond every parameter: id's polymorphic behavior (run id picks the failed or last job and reads the latest attempt), the cost differential between grep versus stepKey/stream, the paging semantics that give pageToken and tailLines their meaning, and the context-cost warning on includeTimestamps ('Costs context; usually not needed'). These semantics are required to invoke the tool correctly and reside only in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening line 'Fetch a bounded slice of the persisted logs for a Depot CI job attempt' names a precise verb, resource, and scope, and the 'bounded slice' qualifier distinguishes this from a generic 'get all logs' tool. It also explicitly names depot_diagnose_ci_failure as the sibling it is not, so an agent can route correctly without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states unambiguously when to prefer the alternative ('Try depot_diagnose_ci_failure first') with concrete reasons why it is cheaper and what it contains, then gives exact conditions for using this tool ('when you need detail the diagnosis did not include: the full traceback, output from a step that did not fail, or a specific pattern'). This is fully explicit when/to-when-not guidance with a named alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.