Server Quality Checklist

Tool Scores

• get-studio-templatesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It mentions authentication via API key but doesn't disclose whether this is a read-only operation, what format the templates are returned in, if there are rate limits, or if it requires specific permissions. The description is functional but lacks important operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a simple retrieval tool, though it could be slightly more front-loaded by mentioning the resource first (e.g., 'Retrieve all available studio templates...').

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one optional parameter and no output schema, the description is minimally adequate. However, it lacks context about what 'studio templates' are, how they differ from library templates, and what the return structure looks like. With no annotations and no output schema, more completeness would be helpful for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single optional 'apiKey' parameter. The description adds no additional parameter semantics beyond what's in the schema (e.g., no clarification on authentication precedence or format requirements). Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'all available studio templates', specifying the scope is for 'the user using their API key'. It distinguishes from siblings like 'get-library-templates' by focusing on studio templates, but doesn't explicitly contrast with other studio-related tools like 'generate-image-from-studio'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose this over 'get-library-templates' for library templates, or when to use it before 'generate-image-from-studio' for generating images from studio templates. No exclusions or prerequisites are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate-imageB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions automatic detection and URL mapping for studio templates, which adds some context, but fails to cover critical aspects like authentication requirements (implied by apiKey parameter), rate limits, error handling, or what the tool returns (only hints at responseType). For a tool with 6 parameters and no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that are front-loaded with the core functionality. Each sentence adds value: the first states the purpose and auto-detection, the second details studio-specific features. There's no redundant information, though it could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, nested objects, no output schema, and no annotations), the description is incomplete. It doesn't explain the output (e.g., what a 'url' response contains), error conditions, or how modifications interact with template types. For a generative tool with multiple parameters and siblings, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing detailed documentation for all parameters. The description adds minimal value beyond the schema, only reiterating that templateId auto-detects library/studio and that modifications work for both types. It doesn't explain parameter interactions or provide usage examples, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate an image from an Orshot template' with the specific action of automatic detection between library and studio templates. It distinguishes itself from siblings like 'generate-image-from-library' and 'generate-image-from-studio' by handling both types automatically, though it doesn't explicitly contrast with all siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool by mentioning it 'automatically detects library vs studio template', suggesting it's a general-purpose alternative to the more specific sibling tools. However, it lacks explicit guidance on when to choose this over 'generate-image-from-library' or 'generate-image-from-studio', or any prerequisites like authentication needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate-image-from-libraryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'specified modifications' but doesn't explain what happens during generation (e.g., processing time, error handling, or output characteristics). For a tool that creates content, this lack of detail on behavior is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part earns its place by specifying the action, resource, and key input aspect. It's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with full schema coverage but no annotations or output schema, the description is minimally adequate. It covers the basic purpose but lacks behavioral details (e.g., what the output looks like, error cases) and usage context. For a generation tool with no output schema, more completeness would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds minimal value beyond the schema by implying modifications are applied to templates, but doesn't provide additional context like examples or constraints. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('generate an image') and resource ('from an Orshot library template'), specifying the source of templates. It distinguishes from 'generate-image' (general) and 'generate-image-from-studio' (different source), but doesn't explicitly mention sibling differentiation. The purpose is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'generate-image' or 'generate-image-from-studio'. It mentions the template source but doesn't explain why to choose library templates over studio templates or other options. No prerequisites or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate-image-from-studioB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions automatic URL mapping and template ID/name flexibility, which adds some context. However, it lacks critical behavioral details such as whether this is a read-only or mutation operation, potential rate limits, error handling, or what happens when data modifications fail. For a tool that generates images with multiple parameters, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that front-load the core purpose. The first sentence covers the main functionality, and the second adds useful clarification about template identification. There's no wasted text, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what the tool returns (e.g., image data, error responses), behavioral constraints, or how it differs from sibling tools. While the schema covers parameters well, the overall context for an AI agent to use this tool effectively is lacking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds minimal value beyond the schema by mentioning 'Automatically maps URLs to appropriate image fields' (hinting at data parameter usage) and 'template ID (numeric) or template name' (clarifying templateId). This meets the baseline of 3 since the schema does the heavy lifting, but the description doesn't significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate an image from an Orshot Studio template using specified data modifications.' It specifies the verb ('Generate'), resource ('image from an Orshot Studio template'), and mechanism ('using specified data modifications'). However, it doesn't explicitly differentiate from sibling tools like 'generate-image' or 'generate-image-from-library' beyond mentioning 'Studio template'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'Orshot Studio template' and mentioning template ID or name, which suggests when to use this tool. However, it doesn't provide explicit guidance on when to choose this over alternatives like 'generate-image' or 'generate-image-from-library', nor does it mention any prerequisites or exclusions beyond the optional API key.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-library-templatesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions authentication via API key but doesn't describe what 'Get all available' entails—whether it returns a list, paginated results, error conditions, or rate limits. For a read operation with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It front-loads the core purpose ('Get all available library templates') and adds necessary context about authentication. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and authentication but lacks details on return format, error handling, or usage context relative to siblings. For a simple read tool, this is borderline viable but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single optional 'apiKey' parameter. The description adds marginal value by noting it's for authentication and optional if set in environment, but this is largely redundant with the schema's description. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('all available library templates'), specifying it's for the user using their API key. It distinguishes from sibling 'get-studio-templates' by focusing on library templates, but doesn't explicitly contrast with other siblings like 'generate-image-from-library'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'get-library-templates' over 'get-studio-templates' or 'generate-image-from-library', nor does it specify prerequisites or exclusions beyond the implied authentication context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-template-modificationsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions authentication via apiKey (optional if set in environment) and template handling, but doesn't describe the return format (e.g., list of modifications, JSON structure), error conditions, rate limits, or whether it's a read-only operation. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences, front-loaded with the core purpose and followed by a clarifying detail about template handling. There is no wasted language, and every sentence contributes useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool with 3 parameters and behavioral complexity. It lacks details on what 'modifications' means in the return value, error handling, or operational constraints, making it inadequate for full agent understanding without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (apiKey, templateId, templateType) with descriptions and enum values. The description adds minimal value beyond the schema by noting that templateId can be ID or name for studio templates, but doesn't provide additional syntax or format details. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get available modifications') and resource ('for a specific template'), distinguishing it from sibling tools like get-library-templates or get-studio-templates. However, it doesn't specify what 'modifications' entail (e.g., editing options, versions, customizations), leaving some ambiguity about the exact output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying it works for both library and studio templates and mentions ID or name for studio templates, providing some context. However, it lacks explicit guidance on when to use this tool versus alternatives like get-library-templates or get-studio-templates, and doesn't mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check-api-statusA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions authentication validation but doesn't describe what happens during the check (e.g., what tests are performed, what response indicates success/failure, rate limits, or error conditions). For a connectivity validation tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple tool and front-loads the core purpose immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple connectivity check tool with 100% schema coverage but no annotations and no output schema, the description is adequate but has clear gaps. It covers the basic purpose but lacks details about behavioral aspects, response format, and error handling that would be helpful for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter. The description adds marginal value by mentioning authentication validation context but doesn't provide additional syntax, format, or usage details beyond what the schema provides. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Check') and resource ('API connectivity'), and distinguishes it from sibling tools focused on image generation and template retrieval by focusing on API health validation. It provides a complete verb+resource+scope combination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (validating API connectivity and key) but doesn't explicitly state when to use this tool versus alternatives or when not to use it. It provides clear functional purpose but lacks explicit comparative guidance with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.