Server Quality Checklist

Tool Scores

• export_widgetA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint=false annotation, the description discloses important behaviors: it can write a file to disk, defaults to '<widget-name>.widget' when path is 'auto', and omitting path skips writing (implied by 'also write'). It also notes the file is importable at widgets.chatkit.studio. This adds useful context, though it doesn't mention overwrite behavior or permissions, so it is strong but not perfect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise—two sentences that immediately state the purpose, then provide key usage details. Every clause carries essential information: the format, the required input, the optional path behavior, and the default filename. There is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters and a straightforward export function, the description covers the main workflow: pass generationId, optionally provide a path, and know the default filename. The return behavior (file contents) is only implied in the description but is explicitly stated in the schema's path description. Given the availability of that schema, the description is adequately complete, though it could explicitly state what the tool returns when no path is given.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides detailed descriptions for both parameters (generationId and path), covering their meaning and special values. The description essentially restates this information without adding new parameter semantics. Since schema coverage is 100%, a baseline score of 3 is appropriate; the description adds no extra clarity beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Export a generated widget as a .widget file'), which clearly states the tool's function. It also distinguishes this from siblings by referencing the Widget Studio editor's format and the import location, making it unique from generate_widget and display_widget.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: it is meant to be used after generate_widget, since it instructs to 'Pass the generationId from a generate_widget result.' It also provides guidance on the optional path parameter, recommending it for local sessions and explaining the 'auto' behavior. However, it does not explicitly mention when not to use it or contrast with display_widget, so it earns a 4 rather than a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• display_widgetA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already declares this as read-only; the description adds that it re-renders instantly and takes a complete widget node tree, but does not disclose potential error behavior or edge cases. This matches the calibration example where annotations lower the burden and a score of 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences with no redundant information. Every phrase earns its place, and the alternative tool is mentioned efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple display tool with two well-documented parameters and a clear description, all necessary context is provided. The description explains purpose, usage, and data format; no output schema is needed for a display operation, and the read-only annotation covers safety.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds meaningful context beyond the schema by specifying the widget must be a 'complete widget node tree' from a previous generate_widget result and that small edits (fix label, change color) are supported, enhancing the semantic understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool re-renders a widget JSON tree from a previous generate_widget result, with specific use cases (as-is or small edits). It distinguishes itself from siblings by explicitly saying 'For new widgets or layout changes, prefer generate_widget.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use context: from a previous generate_widget result, and with small edits. It also names the alternative tool for new widgets or layout changes, giving clear guidance on when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_widgetA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses many behavioral traits beyond the annotations: it takes 20-60 seconds, the service is stateless, it renders exactly what you provide and invents placeholders for omissions, and it may invent buttons with made-up action types unless 'display only' is specified. These notes enrich the openWorldHint annotation and add practical expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet dense, with every sentence earning its place. It starts with the core purpose, then a prompt recipe, critical caveat about buttons, timing, and iteration guidance. It is well-structured and front-loaded, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description covers all necessary context: the purpose, how to craft input, timing, statelessness, iteration behavior, and when to prefer display_widget. It is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Though the schema fully covers the single 'prompt' parameter, the description adds extensive guidance on how to compose the prompt: naming the widget kind, enumerating all real data, adding layout hints, and the 'display only' flag for static widgets. This goes far beyond the schema's one-line description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Generate a rich inline UI widget from a natural-language prompt using the Widget Studio designer.' It also positions this as 'the primary way to create widgets,' distinguishing it from the sibling tools, especially display_widget which is explicitly mentioned for tweaks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance and contrasts with alternatives: 'This is the primary way to create widgets,' and 'for tiny tweaks, edit the returned widget JSON and use display_widget instead.' It also gives conditional advice for static widgets and iteration, making the usage context crystal clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.