Server Quality Checklist

Tool Scores

• create_folderC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool creates a folder, implying a write operation, but doesn't address permissions, error handling, or what happens if the folder already exists. This leaves significant behavioral gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part of the sentence contributes directly to understanding the tool's function, making it appropriately concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral aspects like permissions, error cases, or return values, which are crucial for an agent to use this tool effectively in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter. The description adds no additional parameter semantics beyond what's in the schema, such as format constraints or examples, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create a new folder') and resource ('within the generated-resumes directory for organizing resumes'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_folders' beyond the creation aspect, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, when not to use it, or how it relates to siblings like 'list_folders' or 'create_resume_template', leaving the agent without contextual usage cues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_resume_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states 'create', implying a write operation, but doesn't disclose behavioral traits such as whether this requires permissions, if it's idempotent, what happens on conflicts, or the format of the output. This is a significant gap for a creation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action, though it could be slightly more structured by including key details like output format or usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool involves creation (a mutation) with no annotations and no output schema, the description is incomplete. It lacks details on what the created template looks like, how it's stored, error conditions, or any behavioral context, making it inadequate for safe and effective use by an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the parameter 'templateNumber' documented as 'Template number (1-10)'. The description adds no additional meaning beyond this, such as explaining what each template number represents or how the placeholder data varies. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'create' and the resource 'template resume structure with placeholder data', making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'generate_resume' (which might create actual resumes vs. templates) or 'create_folder' (which creates folders rather than resume templates), so it misses full sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'generate_resume' or 'create_folder'. It implies usage for creating templates but doesn't specify contexts, prerequisites, or exclusions, leaving the agent with minimal direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_resumeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool can save to custom folders and create them if needed, which adds some context. However, it fails to disclose critical behavioral traits such as whether this is a read-only or mutation operation (implied mutation from 'Generate'), potential side effects (e.g., file creation), error handling, or performance considerations like rate limits. The description is too sparse for a tool that creates files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that efficiently convey the core functionality and a key feature (folder saving). It is front-loaded with the main purpose. There is no wasted verbiage, though it could be slightly more structured by separating usage notes, but this is minor.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters with nested objects, no output schema, and no annotations), the description is incomplete. It lacks details on output (e.g., what the PDF response looks like, error formats), behavioral expectations, and integration with sibling tools. For a tool that generates files and interacts with a directory structure, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the three parameters (resumeData, filename, folderPath). The description adds minimal value beyond the schema: it implies that resumeData is used for PDF generation and mentions folderPath functionality ('saves to custom folders within the generated-resumes directory'). However, it doesn't provide additional semantic context, such as examples of resumeData structure or usage tips, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a resume PDF using the LaTeX Resume API' with the specific action 'Provide resume data and get a professionally formatted PDF.' It distinguishes from siblings like 'create_resume_template' by focusing on PDF generation rather than template creation. However, it doesn't explicitly differentiate from all siblings (e.g., 'create_folder' or 'list_folders'), keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal usage guidance: it mentions saving to custom folders but lacks explicit when-to-use instructions, prerequisites, or alternatives. For instance, it doesn't clarify when to use this tool versus 'create_resume_template' or how it relates to folder management tools like 'create_folder' and 'list_folders'. This leaves the agent with insufficient context for optimal tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_foldersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool lists items, implying a read-only operation, but does not address potential behavioral traits such as pagination, sorting, error handling, or whether it recursively lists subdirectories. This leaves significant gaps in understanding how the tool behaves beyond basic listing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the tool's purpose without unnecessary details. It is front-loaded with the core action and resource, making it easy to parse. However, it could be slightly more structured by separating purpose from usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose but lacks details on behavioral aspects and output format, which are important for a listing tool. The absence of an output schema means the description should ideally hint at return values, which it does not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'path' fully documented in the input schema. The description does not add any additional meaning or context beyond what the schema provides (e.g., it does not explain path formatting or constraints). With high schema coverage, the baseline score of 3 is appropriate, as the description does not compensate but also does not detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List all folders and files') and target resource ('generated-resumes directory'), providing a specific verb+resource combination. However, it does not explicitly distinguish this tool from potential sibling tools like 'create_folder' or 'generate_resume', which serve different purposes (creation vs. listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a purpose clause ('to help with organization'), which implies a general context but does not provide explicit guidance on when to use this tool versus alternatives. No when-not-to-use scenarios or specific prerequisites are mentioned, leaving usage decisions largely to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.