Server Quality Checklist

Tool Scores

• lathe_finishA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for disclosing behavior. It mentions the default output folder and subfolder option, but does not disclose potential overwriting behavior, the need for an existing preview, or file naming details beyond the schema. This leaves significant behavioral gaps for a tool that writes files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the primary action ('Export the current preview as a PDF') and then adds a brief, relevant detail about default destination and folderPath. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, with two optional parameters fully described in the schema and no output schema required. However, the description omits important context like what happens if no preview exists, whether files are overwritten, and the exact filename format (though the schema covers the date suffix). These gaps make it only partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% parameter descriptions: 'filename' (base name with date appended) and 'folderPath' (optional subfolder). The main description reinforces folderPath usage, adding contextual value beyond the schema. This exceeds the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Export the current preview as a PDF.' This is a specific verb ('export') and resource ('current preview'), and it distinguishes itself from siblings like lathe_preview, which presumably creates the preview, by focusing on finalizing/exporting it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by referencing 'the current preview' and the output folder, but does not explicitly state when to use this tool versus alternatives or provide any when-not-to-use guidance. The folderPath parameter is explained, but there is no mention of alternative tools for similar tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lathe_add_templateA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosure. It reveals the action (save), destination (templates folder), and content style (Handlebars), but does not mention overwrite behavior, validation, permissions, or return values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and includes a relevant example. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with complete schema documentation, the description is adequate. It covers the core action and content expectations. Missing details like overwrite behavior are not critical given the schema's completeness and the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters fully described. The description adds Handlebars examples but does not introduce additional semantic meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Uses a specific verb 'Save' and a clear resource 'HTML template into the templates folder'. The description clearly distinguishes the tool from siblings like lathe_templates (which likely lists templates) and lathe_preview (which likely previews them).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for saving new templates but does not explicitly mention when not to use it or point to alternatives like lathe_templates for listing. Usage context is clear but exclusions are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lathe_templatesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the responsibility of disclosing behavior. It indicates a read-only operation ('List') and specifies the source folder, but does not mention potential side effects, permissions, return format, or edge cases. The disclosure is adequate for a simple list operation but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately states the action and target. It is front-loaded and contains no filler or unnecessary details, earning every word's place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no params, no annotations, no output schema), the description is mostly complete. It explains the basic purpose but could optionally mention what the returned list contains (e.g., names, paths, metadata) to fully cover expected output. However, for a straightforward listing tool, the current description is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is trivially 100%. The description adds no parameter information because none is needed. Per the rubric, a baseline of 4 is appropriate when there are no parameters to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: listing resume templates available in the Lathe templates folder. It uses a specific verb ('List') and identifies the exact resource and location, distinguishing it from siblings like lathe_add_template or lathe_preview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings, nor any alternatives or exclusions. The usage context is only implied by the verb 'List' and the resource name, but no explicit direction is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lathe_shapeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses a key behavioral trait: 'The browser window auto-refreshes so the user sees the change immediately.' However, with no annotations provided, the description carries the full burden and omits other side effects like persistence or error behavior, limiting transparency to moderate depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states the main purpose, the second covers behavior and usage. Every sentence earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given its simplicity (two parameters, no output schema), the description adequately covers purpose, usage, and a key behavior. It could mention error handling or return behavior, but such details are not essential for a basic push/update tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters, so the description adds limited additional meaning. It mentions 'optionally a different template,' which aligns with the template parameter but doesn't go beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Push updated resume data (and optionally a different template) into the live preview.' The verb 'push' plus the specific resource 'live preview' distinguishes it from sibling tools like lathe_templates or lathe_preview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Use this after every feedback iteration,' which tells the agent when to invoke the tool. It doesn't mention when not to use it or describe alternatives, but the given guidance is direct and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lathe_previewA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden. It goes beyond basic expectations by revealing that the tool opens a persistent live browser window and that the window refreshes on subsequent lathe_shape calls. This gives the agent useful behavioral context without contradicting any annotation (none provided).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, front-loaded with the core purpose. Every phrase earns its place, with no filler or repetition of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple preview tool with no output schema, the description covers purpose, inputs, and behavioral nuance (refresh behavior). The inclusion of sibling tool context via the template source is sufficient for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds relational meaning: the 'template' parameter is explicitly tied to lathe_templates, and 'data' is described as 'the shape your template expects'. This enriches understanding beyond the schema's bare descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Open a live browser window showing a rendered resume') with a specific verb and resource. It distinguishes itself from siblings by explicitly mentioning 'live browser window' and how subsequent lathe_shape calls refresh it, which sets it apart from lathe_templates or lathe_finish.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the user what to pass ('structured resume data' and 'template name from lathe_templates'), and explains the interaction with lathe_shape for refresh behavior. This provides clear context for when to use it, though it does not explicitly state when not to use alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.