Server Quality Checklist

Tool Scores

• get_ir_viewer_compare_urlB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It adds useful context: short shas are accepted directly, no full-sha lookup is needed, and this viewer is separate from the dashboard's ir-text-viewer. However, it does not disclose whether the returned value is just a URL string, whether commit existence is verified, or any error/fallback behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences with the core operation up front and a practical clarification second. Every sentence contributes value and there is no filler or repetition of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a straightforward URL-builder tool: the schema covers all parameters and the description explains the key behavior. However, with no annotations and no output schema, the description could be more complete by explicitly stating the output is a URL and by mentioning when to prefer sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description only reinforces the short-sha behavior that the schema already documents and adds no new parameter-level meaning beyond what the properties already say.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action (build) on a clear resource (compare link on ir-viewer.furiosa.dev) for base vs current commits, and clarifies this is distinct from the dashboard's ir-text-viewer. It does not explicitly differentiate from sibling tools such as get_ir_viewer_url or compare_commits, though 'compare link' implies the core use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives like get_ir_viewer_url or compare_commits. The description implies a compare use case but does not state exclusions or direct the agent to a sibling for single-commit links or textual diffs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_summariesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does describe the summary field as a JSON blob with compile-time and cycle-count metrics, and mentions pagination. But it leaves out response shape, ordering, error behavior, rate limits, and authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no fluff: the main verb and resource are first, the return detail comes second, and filters/pagination finish. It is adequately sized, front-loaded, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is good for a 7-param, read-only listing operation, but there is no output schema and no annotations, so it should provide more context about the response structure beyond the summary field. It also lacks guidance about when to choose this tool over siblings, leaving part of the decision to the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the description does not need to add much. It does group the filter options meaningfully (exact test name, substring, cloud/context) but the individual parameters are already well documented in the schema. No genuinely new parameter-level semantics are introduced.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the exact action: listing compiler benchmark summary rows for a commit short-sha. It also describes the output content, filters, and pagination, which makes it easy to distinguish from sibling tools like list_recent_commits or URL builders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It communicates that the tool is for listing rows for a given commit and optionally filtering, which implies its usage context. However, it never explicitly says when to reach for this tool versus get_summary or compare_commits, and no alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_summaryA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It does disclose that this is a fetch/read operation and that the response includes a full summary JSON blob, but it says nothing about not-found behavior, authentication, rate limits, or error handling. Adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler. It states the action, resource, and return detail efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter get-by-id tool with no output schema, the description covers the essential call purpose and return content well. It lacks explicit error or edge-case behavior and sibling routing, but the low complexity keeps this gap minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already fully documents the single required 'id' parameter. The description's mention of 'UUID id' aligns with the schema but adds no new meaning beyond confirming that the parameter is the lookup key.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch'), the resource ('one summaries row'), and the lookup criterion ('by its UUID id'). It also mentions the distinctive return content ('full summary JSON metrics blob'), which helps distinguish it from list-oriented siblings like list_summaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a targeted single-row lookup by id, which is clear enough for a simple get-by-id tool. However, it does not explicitly say when to prefer this over list_summaries or mention any exclusions or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tactic_debug_urlA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry behavioral disclosure. It does convey that the tool constructs a link rather than performing a comparison, and that short shas are accepted at the input boundary. It does not state whether it validates commit existence, performs any network activity, or what exact string is returned, but for a URL builder this is a minimal viable level of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, both informative: the first states the tool's action, and the second adds a practical input detail. No filler or redundant phrasing is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a simple URL builder, the full parameter schema covers all invocation semantics, and the description explains the core action, the definition is nearly complete. The main gap is that it does not help an agent choose between this and the other compare/viewer URL tools, but that is partly addressed by the specific tactic-debug domain in the title.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so every parameter already has explicit meaning in the input schema. The description's only parameter-related addition is 'Accepts short-shas directly,' which reinforces the sha format but does not fundamentally add semantics beyond the schema's minLength/maxLength and 'short-sha' descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Build'), a specific resource ('tactic-debug compare link'), and the input relationship ('base vs current commit'). The title additionally names the exact dashboard domain, and the tool is clearly distinct from sibling URL builders like get_ir_viewer_compare_url and get_tc_viewer_url.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the obvious use case—building a tactic-debug compare link for two commits—and adds a practical constraint ('Accepts short-shas directly'). It does not, however, explicitly say when to choose this tool over sibling tools such as get_ir_viewer_compare_url or compare_commits, so the when/when-not guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_commitsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds meaningful behavior: results are newest first, and the returned sha must be truncated to ten hex characters before use in list_summaries. It doesn't fully describe the return shape, but this is still strong for a simple list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, main purpose first, critical downstream caveat second. There is no filler, no repetition of schema details, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple optional-limit list operation, the description covers source, ordering, and the one non-obvious transformation an agent must apply (sha truncation). The lack of an output schema is partially mitigated by naming the returned sha field, though the full set of returned fields is not specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, limit, is fully described in the schema with default, minimum, maximum, and meaning, so schema description coverage is 100%. The tool description adds no parameter-specific detail, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List'), a specific resource ('recently benchmarked commits from master_commits'), and ordering ('newest first'). This makes the tool's purpose unambiguous and distinguishes it from sibling tools that deal with summaries or comparisons.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It offers no explicit when-to-use guidance or alternatives. The only workflow hint is the downstream instruction to truncate the sha before using it in list_summaries, but it never says when to choose this tool over siblings like list_summaries or compare_commits.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ir_viewer_urlA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses a non-ovious internal behavior: short-sha resolution to full commit sha via master_commits. It also clarifies that the tool does not run anything locally, which is useful behavioral context for an agent. It does not mention error cases or output format, but the 'link' outcome is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The main purpose is front-loaded, and the additional sha-resolution detail is placed in the second sentence. Every clause adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple URL-builder tool with no output schema and no annotations, the description is fairly complete: it states the purpose, the inputs, and one non-previous internal step. It could be more complete by explicitly naming sibling alternatives or stating failure modes, but the essential information for an agent to invoke it correctly is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds a small but meaningful semantic detail: the short-sha is resolved to the full commit sha required by the viewer URL, explaining why the sha parameter has min/max lengths. The test and context parameters are referenced in passing but are already fully described in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Build'), the resource ('dashboard.furiosa.dev/ir-text-viewer link'), and the required inputs ('commit + context + test'). It also distinguishes the tool's job—producing a browser-openable URL rather than executing local work—from the related URL-builder siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use it ('when you want a browser-openable IR text viewer link instead of running locally'), but it does not explicitly mention alternatives or when not to use it, even though sibling tools like get_ir_viewer_compare_url and get_tc_viewer_url exist. This is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tc_viewer_urlA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It communicates that this is a URL-building operation and clarifies the parameter expectations, but it does not state the exact return value shape, whether any network/auth is involved, or possible failure modes. A bit more explicit behavioral detail would be valuable given the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The core action is front-loaded, and the second sentence immediately provides the most useful differentiating detail. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity URL-construction tool with a complete schema and no output schema, the description gives what an agent needs to select and call the tool: what it builds, which commit params it uses, and how it differs from other compare viewers. The main gap is that 'the other two compare viewers' are not named explicitly, though the sibling tool list supplies that context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all four parameters at 100% coverage, so the baseline is 3. The description adds meaningful semantic distinction by warning that context/test must be a single value (not comma-paired) and that baseSha/currentSha are separate params, which prevents a common misuse pattern across sibling compare viewers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Build a tc-viewer compare link') and identifies the exact resource (a dashboard.furiosa.dev/tc-viewer URL). It also frames the tool as a base-vs-current commit comparison, which distinguishes it from URL/recipe builders like get_ir_viewer_url and build_ir_view_recipe_url.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly contrasts tc-viewer with 'the other two compare viewers' by explaining its parameter shape: a single context/test value and separate sha/baseSha params. This gives decision-relevant context for choosing among compare tools, though it does not name the sibling tools explicitly or provide a precise 'use X instead when...' rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• build_ir_view_recipe_urlA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure. It explicitly warns that this is a draft/proposed contract, that the dashboard does not read the recipe param yet, that the tool only validates and encodes, and that the returned recipe field is the only artifact to hand to a future renderer implementer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence earns its place: draft status, intended use, tool behavior, and the inert-recipe warning are all essential. It is front-loaded with the DRAFT warning, though the dense paragraph style could be slightly more scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description provides all critical context: the contract's draft state, the tool's limited role, how to interpret the response's recipe field, and the relationship to the static dashboard renderer. An agent can safely call the tool without facing hidden surprises.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 9 parameters including enum values and examples. The description adds overall framing about composable view directives but does not add per-parameter semantic detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title and description state a specific action: build a URL for the IR viewer containing a structured recipe. It clearly differentiates from siblings by highlighting that this tool converts loose display requests into a fixed set of composable directives, while other tools like get_ir_viewer_url likely serve stable direct URLs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use this tool: when an AI agent receives a loose display request and needs to translate it into a structured recipe rather than deriving custom rendering from raw IR data. It does not explicitly name alternatives or say when NOT to use it, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_commitsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly discloses what the tool reports: added/removed tests, pass/fail status changes, top N regressions/improvements for both execution-style and compile-style tests, and a generic fieldChanges section. It also explains that fieldChanges 'flattens and ranks every numeric field,' making the tool's behavior highly predictable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every clause earns its place: core behavior, output sections, ranking explanation, filter semantics, and a usage hint. It is front-loaded with the main diff operation and avoids repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex 7-parameter tool with no output schema and no annotations, but the description explains the full comparison behavior, the sections returned, the ranking logic, filter options, and how to approach the results. Nothing essential for calling the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds useful cross-tool meaning by stating the optional filters are 'same as list_summaries' and by clarifying that baseSha/currentSha are short-shas and topN applies per regression/improvement category. It does not redefine every property, but it enriches the schema meaning enough for a bump.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Diff all matching benchmark rows... between a base and current commit short-sha.' It clearly states what is compared and describes the unique output sections, which distinguishes it from the list/get sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool and even provides an internal guidance hint: 'use fieldChanges as a first-pass scan... before drilling into a specific metric.' It also references list_summaries for filter semantics. However, it does not explicitly state when-not-to-use it or name an alternative tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.