Qualtrics MCP Server

by yrvelez

Overview Schema Related Servers Score Discussions

TypeScript

Local

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.0.0
Disambiguation5/5
Each tool targets a specific resource or action with clear descriptions. Overlapping tools like 'export_responses' and 'export_responses_filtered' are explicitly differentiated by filtering capabilities. The helpers (e.g., 'add_multiple_choice_question') are clearly documented as simplified wrappers, so an agent can easily distinguish them.
Naming Consistency5/5
Tool names consistently follow a verb_noun snake_case pattern (e.g., list_surveys, create_survey, delete_question). Exceptions like 'add_embedded_data' and 'piped_text_reference' are still systematic within their context. The naming is highly predictable.
Tool Count2/5
At 112 tools, the number is extremely high for typical MCP usage. While the server aims to cover a broad API, such a large surface can overwhelm agents and lead to selection difficulty. Most tools earn their place for the domain, but the count exceeds even the 'too many' threshold of 25+.
Completeness5/5
The tool set covers nearly every aspect of the Qualtrics API: CRUD for surveys, questions, blocks, flows, responses, distributions, contacts, quotas, libraries, and more. It includes import/export, translation, and an escape hatch ('qualtrics_api_request') for any missing endpoints, ensuring no dead ends.
Average 3.8/5 across 112 of 112 tools scored. Lowest: 2.9/5.
See the Tool Scores section below for per-tool breakdowns.
- 7 of 7 community issues answered or closed in the last 6 months
- 4 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
Add a LICENSE file by following GitHub's guide. Once GitHub recognizes the license, the system will automatically detect it within a few hours.
If the license does not appear after some time, you can manually trigger a new scan using the MCP server admin interface.
MCP servers without a LICENSE cannot be installed.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The destructiveHint annotation signals destructiveness, but the description adds no further behavioral context (e.g., irreversibility, side effects on related data). The confirmDelete parameter implies a safety measure, but this is not explained in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence, but it is too brief to be truly helpful. It lacks structure (e.g., no explanation of confirmDelete's purpose) while still being minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the destructive nature, two required params, and no output schema, the description is incomplete. It does not mention return behavior, error conditions, or consequences, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and both parameters have clear descriptions. The tool description adds no additional meaning beyond what the schema already provides, which meets the baseline expectation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and resource (distribution), directly matching the tool's name. It is distinct among sibling delete tools targeting different resources, though it could be more specific about what a 'distribution' is.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., other delete tools) or prerequisites (e.g., distribution must exist, impacts on surveys or contacts). The agent must infer usage from context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Only states 'Delete', consistent with destructiveHint=true, but lacks details on consequences, reversibility, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, but underspecified for a destructive action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's destructive nature and lack of output schema, the description fails to provide necessary context like effects on related data or recovery options.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions (100% coverage), so description adds no extra meaning beyond what's already there.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb ('Delete') and resource ('mailing list'), distinct from sibling tools like delete_survey or delete_question.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use or when to avoid, no mention of prerequisites or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond non-destructive hint, no behavioral traits disclosed (e.g., side effects, dependencies). The description adds minimal value over annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single clear sentence, no waste. Could be slightly more structured but efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has 7 parameters and no output schema. Description lacks context about return values, prerequisites (e.g., survey must exist), or error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. The description adds no extra meaning beyond the schema fields; 'rank a list of items' aligns with items parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a rank order question, with specific verb 'create' and resource 'rank order question'. It is distinct from sibling tools like add_multiple_choice_question.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., other add_*_question tools). The description lacks context about prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description provides minimal behavioral context beyond annotations. 'Simplified helper' implies easier use but doesn't disclose any side effects, validation, or required permissions. Since annotations only indicate non-destructive, more transparency is needed for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that is front-loaded with the core purpose. However, it could include more detail without becoming overly long.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and moderate complexity (6 params, 4 required), the description is too brief. It does not explain return values, side effects, or how it differs from the generic create_question tool. Sibling tools include many other question types, so more context would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add any additional meaning beyond what the schema already provides for parameters. It simply repeats the resource type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('create') and the specific resource ('text entry question') with explicit types (single line, multi line, or essay). It distinguishes this tool from sibling tools like add_multiple_choice_question, add_likert_question, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lacks guidance on when to use this tool versus alternatives, no exclusions, no prerequisites. It only mentions 'Simplified helper' but does not elaborate on contexts where it is preferred over other question-creation tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide destructiveHint: false, so description adds minimal context. Does not disclose behavior on duplicate or overwrite, or what the response returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key action. Concise but may be too minimal for full utility.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and description does not mention return value (e.g., response ID). With 3 params and nested objects, more context needed for complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers parameters 100% with descriptions. Description adds no extra meaning beyond schema, so baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Import/create' and resource 'response for a survey'. Distinguishes from siblings like update_response, delete_response, export_responses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Does not explain that it creates a new response, not updates existing, nor compare with other response tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only provide destructiveHint: false. The description adds no further behavioral traits (e.g., confirmation, idempotency, auth requirements), leaving the agent uninformed beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, clear and front-loaded. Could include more detail (e.g., response info) without being verbose, but present structure is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 3 parameters and no output schema, the description omits success response or side effects. Incomplete for an agent to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all parameters. The description adds no additional meaning, meeting the baseline but not exceeding it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create), resource (event subscription/webhook), and context (Qualtrics events). It distinguishes from sibling tools like list_webhooks and delete_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., polling) or prerequisites. The description lacks explicit context for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true. The description adds no additional behavioral context (e.g., response structure, pagination, errors), so it does not enhance transparency beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no extraneous information. While brief, it is efficient and well-structured for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks detail on what 'detailed information' includes, especially given siblings like get_survey_metadata. It does not cover return format or scope, making it incomplete for a tool with moderate complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for both parameters. The description does not add any parameter-specific meaning beyond the schema, resulting in a baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves detailed information about a survey. However, it does not differentiate from siblings like get_survey_metadata or get_survey_options, which also retrieve survey info.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as get_survey_metadata or list_surveys, leaving the agent without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation destructiveHint: true already indicates the tool is destructive. The description adds 'permanently', reinforcing irreversibility, but does not disclose other behavioral aspects like impact on related data or required permissions. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and to the point, with no unnecessary words. However, it could be slightly more informative without losing conciseness, such as noting the requirement for confirmDelete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with three parameters and a destructive annotation, the description covers the basic action but lacks context such as prerequisites (e.g., survey must exist) or side effects. It is adequate but not fully complete given the absence of output schema and sibling differentiation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Since the input schema has 100% coverage with descriptions for all three parameters, the description does not need to add parameter details. However, it also does not provide any additional context beyond what is in the schema, so it meets the baseline but adds no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Permanently delete a quota from a survey' uses a specific verb and resource, making the action clear. However, it does not differentiate this tool from sibling delete tools like delete_survey or delete_quota_group, which have similar purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as deactivating a quota or using other quota management tools. It lacks any context about prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description aligns with the destructiveHint annotation, but adds no extra behavioral context beyond what is already indicated. It does not disclose side effects, permanence, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, using a single phrase with no wasted words. It is front-loaded but could benefit from a slightly more structured sentence to improve readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple destructive operation and full schema coverage, the description is minimally complete. However, it omits details about return values or permissions, which would be helpful for agents.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents all three parameters. The tool description does not add any additional meaning or usage context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a survey response' clearly states the verb and resource, distinguishing it from siblings like delete_survey and delete_question. However, it lacks additional specificity about scope or context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., when not to delete, prerequisites). The destructiveHint annotation hints at caution, but the description itself offers no explicit direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond non-destructive annotation, description only mentions automatic mapping. No disclosure of side effects, permissions needed, or response behavior. Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences, no redundancy. Front-loaded with purpose and key feature.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate given schema coverage, but lacks details on return values, error handling, or how choices map to recode values. Moderate complexity (8 params, nested objects) not fully addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. Description adds no additional parameter-level meaning beyond stating it's a simplified helper with automatic mapping. Baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a multiple choice question and notes automatic mapping to Qualtrics types, distinguishing it from sibling add_*_question tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives (e.g., other add_question tools). Only implicit through the tool name and 'simplified helper' phrasing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Minimal behavioral context beyond the destructiveHint=false annotation. Does not disclose side effects, error conditions, or complexities of the 'raw definition' requirement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two brief sentences that communicate the main action and an optional feature without extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple create operation but lacks details on return values (no output schema) and behavior when adding to a quota group. Could be more complete given the complex nested parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions. The description adds little new meaning beyond the schema, only reiterating the 'raw definition' concept.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'create', the resource 'quota', and distinguishes from sibling tools like update_quota or delete_quota by specifying 'from a complete raw Qualtrics quota definition' and optional group addition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as update_quota or list_quotas. No prerequisites or conditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=false, so description carries burden for additional behavioral traits. Description is minimal and does not disclose key behaviors, such as whether sending is immediate or scheduled based on sendDate, or that it creates a new reminder record. However, schema hints at scheduling, partially mitigating transparency gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single clear sentence with no wasted words. It is front-loaded with the core purpose, but could be slightly longer to include usage guidance without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 9 parameters, no output schema, and nested objects, the description is too brief. It omits prerequisites (e.g., distribution must exist), behavioral details (immediate vs. scheduled), and what the response contains. Schema covers parameters but description fails to provide higher-level context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover 100% of parameters with meaningful explanations. The tool description adds no additional meaning beyond what the schema provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Send a reminder for an existing email distribution' clearly states the action (send) and the resource (reminder for an existing distribution). It distinguishes from siblings like create_email_distribution and create_thank_you.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., create_thank_you), no prerequisites (e.g., distribution must exist), and no exclusions. Sibling tools offer similar sending functionality but the description does not differentiate usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only states the basic action (create a new survey) without disclosing behavioral traits like required permissions, side effects, return behavior, or the initial state of the created survey. Given the lack of annotations beyond destructiveHint=false, more detail is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no fluff, front-loaded with the key action and resource. Every word contributes, making it efficient for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple create tool with a fully described schema, the description is minimally adequate. However, given the large number of sibling tools, additional context about what creating a survey entails (e.g., default settings, initial state) would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter (name, language, projectCategory) having a description. The tool description adds no additional parameter meaning beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new survey in Qualtrics' uses a specific verb ('create') and resource ('survey'), clearly distinguishing it from sibling tools like delete_survey, update_survey, or copy_survey. It also identifies the platform, leaving no ambiguity about the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as copy_survey or import_survey_qsf. There is no mention of prerequisites, context, or exclusions, leaving the agent to infer usage solely from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true. The description adds 'permanently', reinforcing irreversibility. No additional behavioral context (e.g., what happens to linked objects) is provided beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Seven words in a single sentence, no fluff. Front-loaded with key information: 'Permanently delete'. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with full schema and annotations, the description is adequate but lacks context about prerequisites like permission requirements or error states. The confirmation parameter is not highlighted.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all parameters. The description adds no new information about parameters, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'delete' and the resource 'graphic from a Qualtrics library'. It is specific and distinguishes from sibling tools like 'delete_library_message', though it doesn't explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'upload_library_graphic' or other delete tools. The context for usage is implied but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, which the description aligns with by saying 'list'. However, the description adds no extra behavioral info such as pagination behavior, response format, or rate limits. With annotations covering safety, a score of 3 is appropriate for not adding value beyond the structured field.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence with no wasted words. It is appropriately concise for a simple listing tool, though it could include brief notes on pagination or filtering without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description lacks information about the return value (no output schema). For a listing tool, agent needs to know if response is paginated, what the fields are, etc. This gap makes it incomplete for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with clear descriptions for both 'offset' and 'username'. The description does not add any extra meaning beyond what the schema provides, so baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states verb 'List', resource 'users', and scope 'in your Qualtrics organization'. It clearly distinguishes from sibling 'get_user' which targets a single user, and it's not ambiguous with other list tools like 'list_contacts' which reference different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'get_user' for a single user. Does not mention pagination behavior or that it returns a list of users. The presence of an 'offset' parameter implies pagination but description doesn't clarify usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=false, indicating a non-destructive operation. The description adds no further behavioral context (e.g., whether sending is immediate if sendDate is omitted, or permission requirements). It is adequate but not enhanced beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately conveys the tool's purpose. No superfluous words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 11 parameters (8 required) and no output schema, the description lacks a high-level explanation of the process, such as the need for a verified email, the role of the library and message template, or what the tool returns. The schema covers individual params but not the overall workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description does not add additional meaning beyond the schema's parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Send a survey via email to a mailing list' clearly specifies the action (send), the resource (survey), the channel (email), and the target (mailing list). This distinguishes it from sibling tools like create_anonymous_link or list_distributions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., create_anonymous_link, which generates a link for distribution). The description does not mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only provide destructiveHint: false, so description should disclose side effects, permissions, or limits, but it does not. Returns no behavioral context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and resource. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema; description does not indicate return values or error conditions. For a creation tool with 4 parameters, more completeness is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds context about message categories and multilingual nature, but does not significantly enhance parameter understanding beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a reusable multilingual library message for specific categories, distinguishing it from sibling tools like update_library_message and delete_library_message.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description implies creation use but does not mention exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, indicating a safe read operation. The description adds that it returns 'detailed information', which is vague but not contradictory. It does not elaborate on permissions, error conditions, or response structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence without wasted words. However, it is minimal and could include slightly more detail without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema), the description is adequate but not thorough. It doesn't mention expected output fields or any prerequisites, leaving some ambiguity for complex use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a well-described 'userId' parameter. The description adds no additional meaning beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (get) and the resource (detailed information about a specific user). It distinguishes itself from sibling tools like list_users (which lists all users) and other get_* functions targeting different entities (e.g., surveys, questions).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as list_users for finding users or when to supply a specific userId. No when-not or alternative suggestions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation declares readOnlyHint=true, and the description does not add any behavioral details beyond that. It fails to mention pagination behavior (pageSize, skipToken) even though the schema includes these parameters, which could lead to incomplete expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the tool's purpose and scope. Every word earns its place; no redundant or unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only list tool with pagination parameters and no output schema, the description adequately indicates the return content ('complete quota definitions'). It could explicitly mention pagination, but the schema provides enough for an agent to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for all three parameters (surveyId, pageSize, skipToken). The description adds no additional parameter semantics beyond what the schema already provides, so it meets the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action ('List') and the resource ('complete quota definitions for a survey'), and includes details on what is covered (logic, limits, actions, schedules, action options). This effectively distinguishes it from sibling tools like get_quota (single quota) and quota group tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., get_quota for a specific quota, or list_quota_groups). It does not mention any prerequisites, limitations, or context for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description provides no behavioral details beyond 'add a contact to a mailing list'. With only destructiveHint=false in annotations, the description should disclose side effects, duplication handling, or permissions, but does not.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence of 6 words that communicates the core purpose without waste. It is appropriately front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 10 parameters (2 required), nested objects, and no output schema, the description is minimal. It does not synthesize or prioritize parameter usage, leaving the agent to rely solely on the schema. Adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description does not need to explain parameters. However, it adds no extra meaning beyond the schema, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('add') and resource ('single contact to a mailing list'), which distinguishes it from siblings like bulk_import_contacts (bulk add), update_contact (modify), and remove_contact (delete).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for single contacts via the word 'single', but lacks explicit when-to-use or when-not-to-use guidance compared to siblings like bulk_import_contacts. No alternatives are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations only provide destructiveHint: false. The description adds no additional behavioral context beyond 'Create', such as required permissions, side effects, or return behavior. This is insufficient for a creation tool, leaving the agent unaware of potential mutations and outcomes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the action and purpose. Every word is meaningful, and there is no unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (3 parameters, no output schema) and the presence of similar sibling tools, the description is minimally adequate. However, it leaves gaps regarding error handling, idempotency, and return value, which could be improved for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for each parameter. The description adds overall purpose but no parameter-specific semantics. According to the guidelines, with high schema coverage, the baseline is 3, and the description does not elevate beyond that.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create) and the resource (mailing list) with a specific purpose (contact management and survey distribution). This distinguishes it from sibling tools like update, delete, list, and get, which operate on existing mailing lists.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (when a new mailing list is needed) but does not provide explicit guidance on prerequisites, when not to use it, or alternatives. Given the context of sibling CRUD tools, the usage is somewhat clear but could be improved with explicit context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation only indicates non-destructiveness (destructiveHint: false). The description adds little beyond the creation action; it does not disclose behavior like duplicate handling, response format, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the core purpose with no extraneous words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has moderate complexity (2 parameters, one nested object) and no output schema. The description covers input sufficiently but omits return behavior or error scenarios, leaving some gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters are documented in the schema (100% coverage). The description enhances the quotaGroup parameter by listing example fields (Name, MultipleMatch, Public, Quotas, Selected), providing useful context beyond the schema's generic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action (create), resource (quota group), and context (from a complete raw Qualtrics definition). This clearly distinguishes it from sibling tools like create_quota and list_quota_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as create_quota for individual quotas. There is no mention of prerequisites, exclusions, or typical usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation 'destructiveHint: true' already informs the agent that this is a destructive action. The description does not add extra behavioral context (e.g., irreversibility, required permissions), but it does not contradict the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words, efficiently conveying the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple deletion tool, but fails to mention return values or success indicators. Given no output schema, the agent would benefit from knowing what response to expect after deletion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, all three parameters (blockId, surveyId, confirmDelete) are documented in the schema. The description adds no additional semantic meaning beyond what is already provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Remove a block from a survey' uses a specific verb ('remove') and resource ('block from a survey'), clearly distinguishing it from sibling tools like 'delete_survey' or 'delete_question'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Sibling tools exist (e.g., 'delete_survey', 'delete_question'), and the description does not specify conditions or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation already indicates destructiveHint=true. The description adds 'permanently delete' confirming irreversibility, but does not disclose other behavioral traits like permissions or consequences if the message is in use.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that conveys the essential purpose without unnecessary words. It is well-structured for quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with three parameters and no output schema, the description is minimally complete. It doesn't explain return behavior or failure cases, but annotations cover destructiveness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides descriptions for all three parameters (100% coverage). The tool description adds no extra semantic meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action (delete) and the resource (reusable library message). It distinguishes from sibling tools like delete_survey (different resource) and delete_library_graphic (different type).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, prerequisites, or scenarios where deletion is not appropriate. The description only states the action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide destructiveHint=true, and description confirms deletion. No extra behavioral context added, such as irreversibility or the need to set confirmDelete=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no waste. Front-loaded with action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple delete tool with annotations and schema coverage, but missing context about the required confirmDelete flag and permanence.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions. The tool description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete an event subscription (webhook)', specifying the verb and resource. It distinguishes from sibling tools like create_webhook and list_webhooks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Does not mention prerequisites like obtaining the subscriptionId from list_webhooks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, and the description adds no extra behavioral context (e.g., no mention of authentication, rate limits, or effect on data). It simply states the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence of 10 words with no redundancy. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description does not explain what 'detailed information' includes (no output schema), leaving some ambiguity. However, for a simple read operation it is minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description provides no additional meaning beyond what the input schema already specifies. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and resource ('detailed information for one contact in an XM Directory mailing list'), clearly distinguishing it from sibling tools like 'list_contacts' or 'add_contact'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit guidance on when to use this tool versus alternatives, nor does it specify any prerequisites or exclusions. Usage is implied by the function name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, so the description adds no new behavioral context (e.g., pagination, performance, or idempotency). It does not contradict the annotations, but the description is minimal beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, front-loading the core action. It is appropriately sized for a simple list operation, but could be slightly more informative without adding much length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no mention of what is returned (e.g., block IDs or full objects) or any limitations (e.g., maximum block count), the description is incomplete. For a list tool, the return structure is important context that is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema describes surveyId as 'The Qualtrics survey ID' with 100% coverage. The description 'List all blocks in a survey' reinforces that surveyId identifies the survey, adding no additional semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and resource 'all blocks in a survey', clearly distinguishing it from sibling tools like 'get_block' (retrieves a single block) and 'create_block' (creates a block). It states exactly what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used to list all blocks, but it does not explicitly state when to use it versus alternatives like 'get_block' for a specific block. No exclusion criteria or usage context is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description is consistent. However, no additional behavioral details (e.g., pagination behavior, limits) are provided beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, front-loaded sentence with no extraneous information. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimally adequate for a simple listing tool with one optional parameter. Lacks details about return format, error handling, or rate limits, which could be useful given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage with a clear description of the 'offset' parameter. Description adds no extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List Qualtrics libraries', specifying the exact resource and action. Distinguishes from sibling tools like list_library_blocks by being the general library listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as list_library_blocks or list_library_questions. Does not specify context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint: true, so the description does not need to emphasize safety. The description adds context about the types of messages listed and the purpose (finding IDs), which provides behavioral intent beyond annotations. However, it does not disclose pagination behavior or result ordering, which are relevant for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the action and resource. Every sentence serves a purpose: the first states what the tool does, the second gives a usage hint. No redundant or unnecessary words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with 3 parameters and no output schema, the description covers the basic purpose and usage. However, it lacks details on the return format (e.g., fields like messageId, name) and pagination via offset. These are important for an agent to use the tool effectively, leaving some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains each parameter. The description adds minimal extra meaning beyond mentioning 'library' which corresponds to libraryId. It does not elaborate on offset or category, so it does not significantly enhance understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists messages in a library and enumerates types (email, survey, etc.). It also mentions a specific use case (finding messageId for distributions), which adds clarity. However, it does not explicitly differentiate from other list operations like list_library_blocks, though the name and context of message types help.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a specific usage scenario ('Use this to find messageId values for distributions'), giving some guidance on when to use the tool. It does not include when not to use it or mention alternatives like get_library_message for retrieving a single message. This is adequate but not thorough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true and destructiveHint=false. Description adds no additional behavioral context beyond 'Update', such as whether missing contacts cause errors or if partial updates are supported. Bar is lowered by annotations, but description doesn't enrich transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words. It clearly states the action and object. However, it could be slightly expanded to include more context without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 11 parameters, nested objects, and no output schema, the description is insufficient. It lacks details on response behavior, required context (e.g., directoryId meaning), and how the update interacts with mailing lists. Agent would struggle to use it effectively without additional information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all 11 parameters. The tool description adds no parameter-level meaning beyond what the schema already provides. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates an existing contact in a mailing list. It uses a specific verb ('update') and resource ('contact in a mailing list'), and is distinct from sibling tools like add_contact, remove_contact, or list_contacts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies updating an existing contact, but gives no explicit guidance on when to use this tool versus alternatives (e.g., add_contact for new contacts, remove_contact for deletion). No prerequisites or when-not-to-use information is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide idempotentHint=true and destructiveHint=false, so the description adds minimal behavioral context; it doesn't clarify partial updates, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no redundancy; effectively communicates the core action and target.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks information on return value or success indicators; with no output schema, the agent cannot determine what the tool yields after execution.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds no extra meaning beyond the schema's parameter descriptions; baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and the resource ('XM Directory mailing list') with specific attributes ('name or owner'), distinguishing it from create/delete mailing list siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like create_mailing_list or delete_mailing_list; lacks context for selecting among sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide idempotentHint and destructiveHint. The description adds that the tool updates metadata, which is consistent but not additional behavioral insight (e.g., effect on responses, versioning). No contradiction to annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with 10 words, no redundancy, and directly states the tool's function. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but the tool is a simple update. However, the coexistence with update_survey_metadata sibling is not addressed, leaving ambiguity. Given 4 params and annotations, the description is adequate but not fully complete in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for all 4 parameters, each with clear descriptions. The description merely repeats the field types ('name, active status, or expiration') without adding new meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Update' and identifies the resource 'survey metadata' with examples of fields (name, active status, expiration). It clearly differentiates from sibling tools like delete_survey (destructive) and activate_survey/deactivate_survey (single-property toggles).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this vs. specialized alternatives (activate_survey, deactivate_survey) or the similar update_survey_metadata sibling. The description does not state exclusions or context, leaving ambiguity for the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already indicates the tool is read-only. The description adds no further behavioral context, such as what status values are possible or whether repeated checking is expected. It does not contradict the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no redundancy. However, it could be slightly expanded to include the return status without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool (2 required params, no output schema, readOnly), the description provides the minimum viable information. It does not explain what the status response looks like or any polling strategy.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters are fully described in the input schema with minLength and descriptions. The description adds no additional meaning beyond what schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'check' and the resource 'status of a response export job', clearly indicating what the tool does. It distinguishes itself from sibling tools like export_responses (which initiates the export) and download_export_file (which retrieves the file).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage after calling export_responses, but it does not provide explicit guidance on when to use this tool versus alternatives or mention any prerequisites or best practices like polling intervals.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide idempotent and non-destructive hints. The description adds minimal extra context beyond stopping responses; does not mention effect on existing data. With annotations covering safety, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, front-loaded with action and outcome. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with annotations and full schema, the description is nearly complete. Minor gap: no mention of whether responses collected before deactivation are preserved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and describes surveyId clearly. Description does not add parameter details, so baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (deactivate) and the resource (survey) with a concrete effect (stop collecting responses). It distinguishes from activate_survey and delete_survey.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like activate_survey or delete_survey. Missing context about prerequisites (e.g., survey must be active).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description aligns with 'get'. No additional behavioral context is provided, but there is no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with the action and resource, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only one parameter and no output schema, the description is adequate for a simple get operation. A minor improvement would be to hint at the return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions; the tool description adds no extra meaning beyond what's in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'language codes currently enabled for a survey'. It differentiates from siblings like update_survey_languages and get_survey_translations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention when not to use it or provide context for selecting among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, indicating safety. The description adds no additional behavioral context (e.g., pagination, limits), but doesn't contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and resource. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimal but sufficient for a simple list tool with one parameter and no output schema. Could mention versioning semantics but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with surveyId described. Description reiterates 'survey definition' but adds no semantic detail beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('list') and resource ('saved versions of a survey definition'), differentiating it from siblings like list_surveys and get_survey_version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., get_survey_version for a specific version). No usage context or prerequisites provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true. The description adds 'currently defined' which implies current state, but does not disclose any other behavioral traits such as rate limits or output format. It is adequate but does not go beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no unnecessary words. It is front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a simple list operation with one parameter, the description is fairly complete. However, it lacks details about the return value (e.g., what properties are included, pagination) since there is no output schema. Minor improvement possible.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides a description for 'surveyId' ('The Qualtrics survey ID') with 100% coverage. The description does not add any additional meaning or context for the parameter, so it meets the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action ('List all'), the resource ('Web Service elements'), and the scope ('in a survey's flow'). It distinguishes this tool from other list tools like list_surveys and list_questions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives (e.g., add_web_service, delete_web_service). No when-to-use or when-not-to-use information is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only include destructiveHint=false. Description adds that graphics become publicly accessible, a behavioral outcome. But lacks details on rate limits, authorization needs, or consequences of uploading with same name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, concise and front-loaded. Every sentence adds value: purpose, allowed types, sources, and outcome. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complex tool with 7 parameters and no output schema. Description does not explain the return value (e.g., graphic URL or ID) or size limit (10 MB base64 from schema). Missing key information for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter already has a description. The tool description reiterates the two source options (URL vs base64) but adds no new semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool uploads a graphic (JPEG, GIF, PNG) to a Qualtrics library from a public HTTPS URL or base64 bytes. It specifies allowed formats and sources, distinguishing it from sibling tools like delete_library_graphic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Says 'non-confidential' and notes public accessibility, giving some guidance on when to use. However, no explicit when or when-not to use, nor mentions of alternatives among siblings (e.g., other library uploads).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true and destructiveHint=false, so the description doesn't need to repeat safety. It adds the effect of starting response collection, which is consistent. No extra behavioral detail beyond what annotations imply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded. Every word is necessary and contributes to understanding the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a simple tool with one parameter and annotations, but it lacks mention of what the response looks like (common for activation tools) and any prerequisites (e.g., survey must exist). Annotations slightly compensate, but some context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%; the single parameter surveyId is described in the schema as 'The Qualtrics survey ID'. The description does not add any additional meaning or context for the parameter, so baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (activate a survey) and the purpose (to begin collecting responses). It distinguishes from sibling tools like deactivate_survey by specifying the opposite effect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (when you want to start collecting responses) but does not explicitly state when not to use or provide alternatives. No guidance on prerequisites like survey being in draft.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are minimal (destructiveHint=false). The description adds behavioral context about default Likert single-answer and inline text entry, but does not disclose permissions, limitations, or side effects beyond what is implied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three well-structured sentences with front-loaded purpose. Every sentence adds essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (10 params, nested objects) and no output schema, the description covers key aspects (default, variants, inline text) but could mention more parameter behaviors like forceResponse or recodeValues.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by clarifying that statements can be objects for inline text and explains the roles of choices and answers, supplementing the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a matrix question and explains terms like statements and scale points. It mentions default Likert and variants via selector/subSelector, but does not explicitly differentiate from siblings like add_likert_question.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for matrix questions and gives one example of a variant, but lacks explicit guidance on when to use this tool versus alternatives like add_likert_question or add_constant_sum_question.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only provide destructiveHint: false, so the description carries full burden. It does not disclose permissions, reversibility, failure behavior, or side effects beyond creation. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with core action, no redundant information. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers core purpose and key parameters like responseMapping and additionalFields. However, lacks explanation for return values, required parameter referenceFlowId, and overall flow impact. Given complexity and no output schema, more context would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. The description adds value by explaining responseMapping for piped text and additionalFields for advanced options, but other parameters (url, method, placement) are not further detailed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it adds a Web Service element in the survey flow, and distinguishes from siblings like list_web_services and add_embedded_data by specifying the resource and purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives context for using additionalFields for authentication/headers, but does not explicitly state when to use alternatives (e.g., add_embedded_data) or when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation 'destructiveHint=true' indicates mutation, and the description aligns with deletion. However, no additional behavioral context (e.g., irreversibility, impact on survey state) is provided beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is appropriately concise for a simple delete operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (three required params, no output schema), the description adequately explains the core action. A note about irreversibility would be a minor improvement but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all three parameters with descriptions (100% coverage), so the description adds no extra parameter meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Remove') and resource ('question from a survey'), clearly distinguishing it from sibling tools like 'delete_survey' or 'update_question'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs. alternatives. The description implies usage for deleting a single question, but no exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. Description adds detail about content (delivery stats) but no behavioral traits beyond what annotations provide. Consistent, no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single informative sentence, front-loaded, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate but vague on output details; does not mention what 'detailed information' includes beyond delivery stats. No output schema to compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%; both parameters have clear schema descriptions. Tool description does not add additional meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb (Get) and resource (detailed information about a specific distribution including delivery stats). Distinct from siblings like list_distributions (list all) and get_distribution_history (history).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage for retrieving details of one distribution, but no explicit guidance on when to use vs alternatives or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, so safety is covered. Description adds that it returns types and preview text, but does not disclose pagination, sorting, or limits. Adequate but not detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, front-loaded with action. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema; description mentions 'types and preview' but does not specify format or fields. Lack of pagination info may be acceptable for a simple list, but more detail would help agents.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the sole parameter 'surveyId', with a clear description. The tool description adds no further parameter semantics, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'list' and resource 'questions', and specifies returned data (types and preview). It clearly distinguishes from siblings like 'get_question' (single question) and 'create_question'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when/when-not or alternative guidance. Usage is implied by sibling names, but the description does not provide context like 'use this before editing questions' or 'this returns all questions without pagination'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only nature. The description adds context about membership and multiple-match behavior but fails to mention pagination behavior despite pagination parameters (pageSize, skipToken) existing in the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, concise sentence that is front-loaded with the core action and includes relevant details. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, so the description must convey the return value. It mentions 'complete quota-group definitions' but lacks specifics on the structure or fields, leaving the agent partly uninformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% description coverage, so parameters are already documented. The description does not add further semantic detail beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool lists complete quota-group definitions for a survey, including membership and multiple-match behavior, differentiating it from siblings like list_quotas and get_quota_group.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like list_quotas or get_quota_group. The context among siblings implies its purpose, but direct comparisons are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond annotations by specifying that explicit common arguments override matching keys in metadata and that date fields require MySQL datetime format. Annotations already indicate idempotentHint=true and destructiveHint=false.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first provides purpose and structure, second adds a critical format requirement. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, key parameter behaviors, and format constraints. While the tool has 7 parameters and a nested object, the description adequately explains the common fields and metadata interaction. No output schema, but annotations provide safety context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the description adds format constraints for date parameters and explains the metadata object's semantics (raw fields, override behavior), which is meaningful beyond the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update survey-definition metadata' and specifies that common fields are exposed directly while metadata accepts additional Qualtrics fields. It distinguishes the tool as metadata-focused, though it does not explicitly differentiate from the sibling 'update_survey' tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. It mentions the structure of metadata but does not provide context for preferred usage or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only provide destructiveHint: false. The description lacks behavioral details such as permissions, idempotency, or effects on existing fields. It adds minimal transparency beyond stating the operation is safe.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: purpose, placement logic, syntax reference. Each sentence is informative and earns its place, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Explains placement and parameter usage clearly. No output schema is provided, but the tool returns a success status (common). Minor omission: no mention of errors or handling duplicate field names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds a note on piped text referencing, but that is already implied in the schema. No significant extra meaning beyond schema fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Add') and resource ('embedded data fields') and clarifies the scope ('anywhere in the survey flow'). It distinguishes from siblings like 'list_embedded_data' by focusing on addition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides placement guidance ('Place declarations at the beginning, or place assignments after...') and syntax reference. However, no explicit when-not-to-use or alternatives compared to other flow manipulation tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The destructiveHint annotation already signals mutability. The description adds the important behavioral detail that the rest of the flow is not rebuilt. However, it does not disclose potential side effects (e.g., references to the deleted element).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that conveys the core purpose and key behavior with no extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a three-parameter tool with a destructiveHint annotation and no output schema, the description adequately covers the tool's function (delete without rebuild). No critical gaps are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents all three parameters. The description adds no additional parameter-level meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete), the resource (flow element), and specifies it works on both top-level and nested elements without rebuilding the rest. This distinguishes it from sibling tools like delete_survey or delete_block.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when wanting to remove a specific flow element atomically, but does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives like update_flow_element or move_flow_element.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds that deletion requires name confirmation as a safety measure, which complements the destructiveHint annotation. However, it lacks details on irreversibility or authorization needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences that convey the core action and safety measure without extraneous detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool, the description covers the essential behavior. It could mention success/error indicators or constraints, but the existing information suffices given no output schema or nested objects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description's mention of name confirmation aligns with the confirmName parameter but adds no extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a survey', using a specific verb and resource. It distinguishes itself from sibling tools like create_survey or update_survey by focusing on deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention prerequisites, consequences, or scenarios where deletion is appropriate or inappropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, so the read-only nature is known. The description adds that the result includes a contactLookupId, but does not disclose other behavioral aspects like pagination or error handling, which are partially covered by the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loads the action and key output, and contains no redundant information. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only data retrieval tool without an output schema, the description explains the output contents (delivery history with contactLookupId). Pagination is implied by the skipToken parameter described in the schema. It does not mention error cases but overall is sufficient for the agent to understand usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter described (surveyId, distributionId, skipToken). The tool description does not add any new parameter information beyond what the schema provides, so it meets the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves per-recipient delivery history with specific statuses and includes the contactLookupId for identity recovery. It distinguishes from sibling tools by focusing on individual-level history, but does not explicitly contrast with alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that it is the documented way to recover respondent identity for individual-link distributions, giving a clear context for when to use it. However, it does not specify when not to use or list alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint: true already indicates a safe read operation. The description adds context that it returns the 'complete raw definition including its logic and actions', but does not disclose any additional behavioral traits (e.g., potential large response size, permission requirements).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It efficiently conveys the tool's core action and output scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only tool with two required parameters and no output schema, the description provides adequate context about the return value ('complete raw definition, logic, and actions'). This helps the agent understand the response structure, though more detail could be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters described in the schema. The description adds no specific parameter-level details beyond 'quota's definition, logic, and actions', which is high-level. Baseline 3 is appropriate as the schema already handles parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and resource ('quota's complete raw Qualtrics definition'), and mentions including logic and actions, which clearly distinguishes it from sibling tools like list_quotas, update_quota, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies it's for retrieving the full definition of a specific quota, but it does not explicitly state when to use this tool versus alternatives (e.g., list_quotas for summaries, update_quota for modifications), nor does it provide any exclusion or prerequisite guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true. Description adds that it returns 'complete raw Qualtrics definition' and 'quota membership', but does not disclose additional behavioral details such as what 'raw definition' entails or whether there are any side effects. With annotations covering safety, the description adds some context but not extensive behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that is front-loaded with the action and resource. No unnecessary words; every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool (read operation), the description is mostly complete. It mentions the return includes 'complete raw Qualtrics definition' and 'quota membership', which compensates for the lack of an output schema. However, it could be slightly more explicit about the response format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and both parameters have descriptions. The description does not add any additional meaning beyond what is already in the schema, so baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'Get a quota group's complete raw Qualtrics definition, including its quota membership', clearly specifying the action (Get), resource (quota group), and scope (complete raw definition with quota membership). This distinguishes it from sibling tools like list_quota_groups, create_quota_group, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The purpose implies it's for fetching a single group's full definition, but there is no mention of when not to use it or reference to other tools for similar tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, so the description doesn't need to emphasize safety. However, it adds no additional behavioral context (e.g., error handling or return format).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, minimal and direct. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with 2 parameters and no output schema, the description is sufficient. Could optionally mention that responseId follows the 'R_...' format, but schema covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description does not add extra meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get a single survey response' with a specific identifier (response ID). Distinguishes from sibling tools like delete_response and update_response.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., export_responses, list_responses). Usage is implied by the straightforward purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, so the description adds no behavioral traits beyond listing returned metadata. It does not contradict annotations. The addition is moderate, not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the purpose. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with one parameter, the description is complete. It lists the kind of metadata returned, compensating for the lack of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes surveyId as 'The Qualtrics survey ID' (100% coverage). The description adds no further meaning to the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('survey-definition metadata'), and lists example fields (ownership, status, language, lifecycle dates), distinguishing it from siblings like get_survey.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives like get_survey or update_survey_metadata. While listing the metadata fields implies a narrower scope, there is no when-not or direct comparison.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description confirms a read operation. The description adds value by listing what the options object contains, but does not disclose additional behaviors like rate limits or security expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, well-structured sentence that front-loads the main action and lists key components. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only getter with one parameter and no output schema, the description adequately covers the returned object's components. Minor gap: no mention of error conditions or optionality of returned fields.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for surveyId. The description adds no additional semantics beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'complete survey options object', listing specific included settings (navigation, protection, etc.). This distinguishes it from siblings like update_survey_options or get_survey_metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is given. The description lists the returned components, which implies its use case, but lacks direct differentiation from similar getters like get_survey_metadata.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, so the description adds some context about returning a 'complete translation map' for one language. However, it does not disclose additional behavior like pagination, size limits, or error scenarios. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 10-word sentence that is front-loaded with the verb and resource. Every word is necessary, with no wasted space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple read operation with two parameters. It could be slightly more detailed about what the 'complete translation map' contains, but the context and schema provide sufficient understanding. No output schema exists, but the domain is straightforward.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters. The description does not add meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'complete translation map', and the scope 'for one enabled survey language'. It effectively distinguishes from sibling tools like update_survey_translations (write) and get_survey_languages (list languages).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for reading translations but does not explicitly state when to use this tool versus alternatives or when not to use it. No exclusions or context about prerequisites are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With only destructiveHint=false in annotations, the description adds value by warning about JavaScript escaping (avoid `$`). But it does not disclose other behavioral traits such as input validation, size limits, or side effects beyond creation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys purpose and common uses. Minor extra context (e.g., 'simplified helper') could be justified, but overall no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, no output schema, and a moderate complexity, the description covers purpose, common usage, and a key behavioral hint (JS escaping). It could be more complete by noting return value or limitations, but is sufficient for a helper tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description adds minimal extra meaning beyond the schema. The only addition is a JS escaping tip, which is already partially covered in the schema description. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a descriptive text (DB/TB) question, specifying its common uses (instructions, processing screens, HTML content, optional JS). This distinct verb+resource purpose differentiates it from sibling question-adding tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides usage context by listing typical use cases (instructions, processing screens, HTML/Javascript). However, it lacks explicit when-not-to-use guidance or direct comparison to alternatives like other question types.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide destructiveHint=false, and the description confirms it's a helper that creates a question (non-destructive). It adds that it uses preset scales but does not detail other behavioral traits like authorization needs or side effects (e.g., no mention of survey modification). Overall adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with purpose, no wasted words. It could include more detail but remains appropriately concise for a helper tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal and leaves out return value (no output schema), error handling, and prerequisites (e.g., survey existence). Given the tool has 8 parameters including nested objects, a bit more context would be helpful, but the schema covers the parameter details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the parameter descriptions already cover meanings. The description adds slight value by explaining the purpose of preset scales (no manual enumeration), but for other parameters like recodeValues, no extra insight beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it's a simplified helper to create a single-item Likert scale as MC/SAVR. This distinguishes it from siblings like add_multiple_choice_question (generic) or create_question (generic). The verb 'create' and resource 'Likert scale' are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context: a simplified helper for Likert scales with preset scales, avoiding manual enumeration. However, it does not explicitly state when not to use (e.g., for other question types) or mention alternatives, leaving some room for misinterpretation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotation only indicates non-destructive. Description adds key behavioral traits: non-atomic execution and per-contact success/failure reporting. This goes beyond the annotation, though it could include more on rate limits or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single 18-word sentence that front-loads the core purpose and key behavioral nuance (non-atomic, per-contact results). No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description does not outline the response structure beyond 'per-contact successes and failures'. Missing guidance on error handling, failure modes, or prerequisites beyond the required parameters. For a bulk operation with partial failures, this is somewhat incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. The tool description adds contextual details like 'up to 100 contacts' and 'created sequentially', and clarifies directoryId as 'also known as the directory or pool ID', adding value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates multiple contacts, specifies it reuses a single-contact endpoint, and highlights the non-atomic nature with per-contact results. This distinguishes it from add_contact (single contact) and other sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like add_contact or other contact import methods. The description mentions implementation detail (using single-contact endpoint) but does not compare to siblings or advise on scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions what is preserved but does not disclose behavioral traits beyond annotations (destructiveHint=false). It lacks details on the return value, prerequisites, or potential side effects like clone limits or activation status. Annotations are minimal, so description carries burden but adds only partial transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that efficiently conveys the tool's purpose and key behavioral context. It is front-loaded and contains no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (3 parameters, no output schema), the description covers the action and what is preserved but omits return value details and prerequisites. It is adequate but incomplete for an agent to fully understand the tool's behavior without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented. The description does not add additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a complete copy of an existing survey, preserving complex aspects. It distinguishes from sibling tools like create_survey (new) and update_survey (modify) by focusing on copying and preserving settings without individual endpoints.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: when you need to copy a survey while preserving complex question types, scoring, flow, styling, and settings that lack individual mutation endpoints. It provides good context for choosing this over alternatives, though it does not explicitly list when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. The description adds value by explaining the tool returns an honest per-response estimate when expectedResponseCount is omitted, and clarifies that survey metadata doesn't expose response totals. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, followed by essential context. No wasted words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers input behavior well but lacks any details about the output format of the estimate (e.g., size in bytes or string). With no output schema, this omission reduces completeness for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters well. The description adds minimal semantic value beyond restating what the schema says, resulting in a baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool estimates export size using question count and optional expected response count, distinguishing it from other tools. It is a specific verb+resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for estimation before actual export but does not explicitly compare to siblings like export_responses or provide when-not-to-use guidance. It offers context on expectedResponseCount but lacks alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. The description adds what the flow tree includes but does not discuss potential performance or authorization issues.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, focused sentence that efficiently conveys the tool's purpose without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple read-only nature with one parameter and annotation support, the description adequately covers what the tool does and what it returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear parameter description. The tool description does not add extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get the full survey flow tree' with specific components (blocks, embedded data, etc.), distinguishing it from other survey-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or alternatives is provided, but the context of siblings and readOnlyHint implies it is for retrieving flow structures.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds behavioral context beyond annotations (destructiveHint=false) by listing supported syntax elements. However, it does not mention error handling or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single concise sentence that front-loads the main purpose and lists supported features without wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description covers purpose and supported features but lacks details on error handling, return value, prerequisites, or permissions, which would be helpful for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions; the description adds value by specifying the text format as 'Qualtrics Simple or Advanced TXT', but this does not significantly increase understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it creates a survey from Qualtrics Simple or Advanced TXT format, distinguishing it from sibling tools like import_survey_qsf and import_survey_docx.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for importing TXT format surveys but does not provide explicit when-to-use or when-not-to-use guidance, nor alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate destructiveHint=false, which aligns with an insert operation. The description adds that a FlowID is auto-generated if not supplied, but does not disclose other potential side effects (e.g., impact on survey flow logic) or permissions required. Some behavioral insight is provided but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences. The first covers the core purpose and the second adds a key behavior. No redundant or extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema, the description does not mention what the tool returns (e.g., the created element with assigned FlowID). It also lacks details on validation of the element structure. Given the complexity of the nested object parameter, more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds value by explaining the 'exact position' concept and the auto-generation of a collision-free FlowID. This enhances understanding beyond the schema's property descriptions, particularly for the 'element' and 'placement' parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Insert'), the resource ('any Qualtrics flow element'), and the scope ('at an exact position'). It lists specific element types and mentions automatic FlowID allocation, distinguishing it from sibling tools like update_flow_element and delete_flow_element.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for adding new flow elements, but does not explicitly compare with alternatives (e.g., update_flow_element, move_flow_element) or specify prerequisites such as survey state. It provides basic context but no when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, which is consistent. The description adds that it returns one cursor-paginated page, providing useful behavioral context beyond the annotation. No contradictions, and the pagination detail helps an agent understand iteration.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise, front-loaded with the action, and contains no unnecessary words. It efficiently communicates the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does not explain the return format (e.g., array of contacts, nextSkipToken). While the pagination intent is clear, the lack of output details makes it minimally adequate for a tool with 5 parameters. A brief note on the result shape would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description does not need to add parameter details. It does not elaborate on parameters, which is acceptable since the schema already documents them clearly. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List one cursor-paginated page of contacts in an XM Directory mailing list', specifying the verb (list), resource (contacts), and scope (one page, specific mailing list). This distinguishes it from siblings like get_contact (single contact) or list_mailing_lists.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing contacts paginated, but does not explicitly state when to use this tool versus alternatives like get_contact for single contacts or bulk_import_contacts for bulk operations. No 'when not to use' guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation provides readOnlyHint=true, so the description is not burdened with safety info. It adds pagination and filter context, but doesn't disclose edge cases like empty results or rate limits. Adequate given annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that conveys the core functionality. Could be slightly more structured (e.g., front-loading pagination), but no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with good annotations and full schema coverage, the description adequately covers pagination and filter options. No output schema is needed for a list endpoint; the description is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with all 7 parameters documented. The description adds only generic 'optional documented Qualtrics filters', adding marginal value beyond the schema. Baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List one cursor-paginated page of distributions for a survey', using a specific verb and resource. It distinguishes from sibling tools like 'get_distribution' (single distribution) and 'list_distribution_links' (links).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'cursor-paginated page' and 'optional filters', giving context for paginated usage. It does not explicitly exclude alternatives, but the purpose is clear enough for a list tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description simply restates the read-only nature. No additional behavioral details are provided, such as rate limits or scope.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, and the key action is front-loaded. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema), the description fully conveys its purpose and behavior. No missing information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers the single parameter with a description. The tool's description does not add extra meaning beyond 'survey ID', so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool lists all embedded data fields in a survey's flow. The verb 'list' and resource 'embedded data fields' are specific, and it is distinct from siblings like add_embedded_data or list_blocks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The purpose is implied but lacks exclusions or context for selection among sibling list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds value by specifying that it returns 'one cursor-paginated page', disclosing the pagination behavior beyond the schema. However, it does not elaborate on error conditions or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the essential information without redundant or extraneous words. It is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description could have explained the return structure (e.g., list of mailing list objects with pagination token). However, the high schema coverage and annotation compensate somewhat, leaving it minimally adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 5 parameters have descriptions in the input schema (100% coverage), so the description adds no additional semantic value for individual parameters. The baseline of 3 is appropriate as the description does not compensate with parameter-specific details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List'), resource ('mailing lists'), scope ('in an XM Directory'), and includes the pagination detail ('one cursor-paginated page'). This distinguishes it from sibling tools like get_mailing_list or list_directories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing mailing lists but provides no explicit guidance on when to use this tool versus alternatives (e.g., get_mailing_list for a specific list). No when-not-to or prerequisite conditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=false. The description adds compatibility context (legacy vs new experience) which is helpful, but does not disclose other behavioral traits like authentication requirements, rate limits, or side effects. This is adequate given the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences that front-load the purpose and then add key usage context. Every word is necessary, no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the question type and variations well. However, it does not mention the return value (e.g., question ID) or any post-creation steps. Given no output schema, this is a minor gap for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description does not need to add much. The description explains the concept of constant sum and the effect of the 'selector' parameter, but the schema already provides decent descriptions. This is baseline 3 with no significant extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a constant sum question with examples (percentages summing to 100). It distinguishes among siblings like add_multiple_choice_question by specifying the question type. The verb 'create' and resource 'constant sum question' are explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on when to use the default variation (choices/text-entry) vs legacy variations (bars, sliders). However, it does not explicitly advise against using this tool for non-constant sum questions, though that is implied by the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide destructiveHint=false, and the description confirms creation (non-destructive). The description adds value by listing advanced features but does not disclose permissions, rate limits, or side effects beyond that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the action, and every sentence contributes purpose. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (7 params, nested objects, no output schema), the description covers the main purpose and capabilities. It does not detail return values or error states, but these are not required without an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions. The description enhances parameter semantics by listing the Qualtrics block definition features (reference blocks, etc.), giving context to parameters like blockElements and options.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and resource 'block', and explicitly lists supported features (reference blocks, question ordering, skip logic, randomization, loop-and-merge), distinguishing it from sibling tools like list_blocks or update_block.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or when not to use, nor alternatives. However, the purpose is clear, and given that only one create tool for blocks exists, the lack of guidelines is not critical.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate destructiveHint: false, meaning non-destructive. The description adds context about scheduling (sendDate) and that the email targets respondents. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no extraneous information. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 9 parameters, 7 required, and no output schema, the description adequately covers the core action. However, it omits details about response behavior (e.g., what happens after sending) and edge cases like past sendDate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 9 parameters have schema descriptions (100% coverage), so the description adds minimal value. It mentions embeddedData as optional tags, but does not clarify relationship between messageId and libraryId beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (send or schedule) and resource (thank-you email for respondents in an existing email distribution). It distinguishes from sibling tools like create_email_distribution, which creates the distribution itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for sending thank-you emails but does not explicitly state when to use this tool versus alternatives (e.g., create_reminder). No usage context or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description is consistent with readOnlyHint=true annotation but adds no extra behavioral context (e.g., auth requirements, rate limits, or what constitutes 'complete metadata'). With annotations present, baseline score is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, front-loads the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple read tool with annotated readOnlyHint; lacks output schema but description implies a rich metadata object. Missing details on content of metadata could be improved but is not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema covers all 3 parameters with descriptions (100% coverage), so the description adds no additional parameter meaning beyond what schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'Get' and resource 'complete metadata for a mailing list', clearly distinguishing it from sibling tools like create_mailing_list, update_mailing_list, and delete_mailing_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Purpose is clear but does not explicitly state when to use vs alternatives. Agent can infer it's for retrieving metadata of a specific list, but no when-not or alternative tool mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the description does not need to restate read-only nature. The description adds that the tool returns 'choices, validation, and configuration', which is useful behavioral context. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-formed sentence that conveys all necessary information concisely. No unnecessary words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with two required parameters and no output schema, the description adequately explains what the tool returns. It covers the key aspects of the question definition. Could be slightly improved by mentioning the tool is read-only, but annotations already cover that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage with descriptions for both surveyId and questionId. The description does not add new meaning beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the full definition of a specific question, including choices, validation, and configuration. This distinguishes it from sibling tools like list_questions (which only lists questions) and create_question (which creates). The verb 'Get' and resource 'question' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives. It implies usage when a full question definition is needed, but no exclusions or comparisons are provided. Given the many sibling tools, explicit guidance would be beneficial.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint annotation, the description adds important behavioral context: the QSF export may not be accepted unchanged by the import API. This informs the agent of a potential limitation not captured in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise—only two sentences. The first sentence states the core purpose, and the second adds a critical caveat. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with annotations and full schema coverage, the description covers the main purpose and a key behavioral nuance. It does not mention return format or error conditions, but given the lack of output schema, the description is reasonably complete. Minor missing detail: how to obtain the versionId.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes all parameters with 100% coverage. The description adds context about the format parameter (explicitly naming JSON and QSF) but does not provide additional semantic details beyond the schema's enum and descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'saved survey version', and specifies the available formats (JSON or QSF). It distinguishes from sibling tools like create_survey_version and list_survey_versions by focusing on retrieval of a specific saved version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (to retrieve a saved survey version) but does not explicitly state when not to use it or mention alternatives. The caveat about QSF not being guaranteed for import is helpful but doesn't provide guidance on choosing between get_survey_version and other tools like list_survey_versions or import_survey_qsf.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only provide destructiveHint: false. The description adds that Qualtrics fetches the file and that private URLs don't work, but doesn't disclose other behaviors like return value, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two-sentence description that is front-loaded with the core action and quickly addresses key constraint. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of sibling import tools and full schema coverage, the description covers the essential constraints. However, lacking output schema, it does not describe the return value, which could be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all three parameters (name, format, fileUrl). The description adds only the clarification about public URLs, which is partially redundant with the fileUrl parameter description indicating a 'Public HTTPS URL'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (create a survey), the resource (survey), and the allowed input formats (publicly accessible QSF, TXT, or DOCX URL). Distinguishes from sibling import tools that may accept direct file uploads.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly notes that the URL must be publicly accessible and that private/local URLs won't work. This helps agents understand constraints, though it doesn't explicitly mention when to use alternative import tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. Description adds context that blocks are 'templates' for 'copying or recreating', implying no side effects and providing usage intent beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no fluff. First sentence states action and resource; second adds purpose. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with 2 parameters and no output schema, the description covers purpose, resource, and usage context. Could mention pagination behavior or clarify 'library' distinction from survey blocks, but overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters. The description does not add extra parameter meaning beyond what the schema provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'list', resource 'reusable survey blocks in a Qualtrics library', and the purpose of discovering complex block definitions. Distinguishes from sibling 'list_blocks' by specifying 'in a library'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for discovery before copying/recreating but does not explicitly state when to use this tool versus alternatives like 'list_blocks'. No when-not or explicit alternative naming.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds no further behavioral details (e.g., pagination behavior, rate limits), providing only basic transparency beyond what annotations offer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the first sentence front-loading the purpose and the second providing usage context. There is no redundant or unnecessary information, every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (2 parameters, no output schema, good annotations), the description covers the essential purpose and usage context. Minor gaps exist, such as not explaining the return format, but overall it is sufficiently complete for an AI agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both libraryId and offset. The description does not enhance or add meaning to these parameters beyond what the schema provides, so it meets the baseline without adding extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb 'List' and the resource 'reusable survey surveys in a Qualtrics library', and distinguishes itself from sibling tools like list_surveys by focusing on library templates. The use case 'discover complex survey definitions before copying or recreating' further clarifies its unique purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states when to use the tool ('before copying or recreating them through MCP'), providing clear context. However, it lacks explicit exclusions or mention of alternative tools for other scenarios, such as using list_surveys for actual surveys.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. The description adds the meaning of 'event subscriptions (webhooks)' but offers no additional behavioral details (e.g., no pagination info, response format). With annotations covering safety, this is adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It earns its place as a concise statement of purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no parameters, the description is mostly complete. It clearly states what is listed (all event subscriptions). Minor gap: no mention of return format or whether results are paginated, but these are non-critical for a simple list.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist and schema coverage is 100%. Baseline for zero-parameter tools is 4. The description adds no parameter info but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'event subscriptions (webhooks)' with scope 'in your Qualtrics account'. It effectively distinguishes this tool from siblings like create_webhook and delete_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing webhooks but provides no explicit guidance on when to use this vs. alternatives (create/delete). Sibling tools exist, making some differentiation helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true and destructiveHint=false, establishing a safe, non-destructive operation. The description adds scope detail (top-level or nested flow) but does not disclose additional behavioral traits like order changes or error conditions, so it meets the baseline for annotated tools.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 18 words, front-loading the verb and object. Every word is necessary, with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity (4 params, one enum) and no output schema, the description adequately covers the tool's purpose and scope. It could mention the conditional requirement for referenceFlowId, but the schema handles that, and the description is sufficient for an agent to understand the operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description adds no new parameter information. Baseline 3 is appropriate as the description does not enhance understanding of parameter formats or constraints beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool moves an existing flow element while preserving its definition and FlowID. It distinguishes from siblings like insert_flow_element (creates new) and delete_flow_element (removes) by specifying repositioning and non-destructive nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly suggests when to use (to reposition an element) and the sibling context provides clear alternatives. It does not explicitly state when not to use, but the verb 'move' combined with sibling names offers sufficient guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the description's confirmation of a lookup operation adds minimal new behavioral info. The description does not disclose other traits like return format or error behavior, but given the low risk, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the main purpose, and contains no extraneous information. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and simple parameters, the description adequately explains the tool's purpose. It could be improved by briefly hinting at the return format (e.g., examples), but it is complete enough for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters have full descriptions in the schema (100% coverage). The tool description does not add additional semantics beyond the schema, which is adequate for this simple lookup tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool looks up Qualtrics piped text syntax and lists the types of references it returns (question responses, embedded data, etc.). It is distinct from sibling tools that perform survey mutations or data exports.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says 'Use this when you need to dynamically insert values into survey questions,' providing clear context for when to use it. However, it does not explicitly state when not to use or compare to alternatives, though the sibling list makes the distinction obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark destructiveHint=true. The description adds that only the mailing list membership is removed, not the contact itself, which is key behavioral context not covered by annotations. However, it does not mention the confirmDelete safety mechanism.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence conveying the action and scope with no redundancy. Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple removal tool with no output schema, the description adequately explains the scope of the removal. It could mention the confirmDelete requirement, but that is already in the schema. Overall sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description does not need to add parameter details. The tool description does not expand on individual parameters beyond what the schema already provides, meeting the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool removes a contact from a mailing list only, and specifies the contact remains in the XM Directory. This distinguishes it from a full delete and aligns with its purpose among sibling tools like add_contact and update_contact.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for removal from a list without directory deletion, but it does not explicitly state when to use this versus alternatives or provide exclusions. No explicit guidance on prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotentHint=true and destructiveHint=false. The description adds valuable context about the default merge behavior (fetching existing definition, preserving omitted fields) and the write operation (normalized complete flow tree). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences that efficiently convey the core purpose and the key default behavior. No unnecessary words or repetition. Front-loaded with the action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main behavior (merge vs replace) and the write operation. However, it does not explain the return value (no output schema) or potential error conditions. For a tool with four parameters, the description is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the description adds minimal extra meaning. The 'element' parameter is described briefly, but the schema already provides detailed descriptions. No additional clarity on 'replace' or 'surveyId' beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('update') and resource ('flow element') with a unique identifier ('FlowID'). It clearly distinguishes from sibling tools like 'insert_flow_element' or 'delete_flow_element' by focusing on updating an existing element.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (to update an existing flow element) but does not explicitly state when not to use it or provide alternatives. It lacks contextual guidance on prerequisites or when to set the 'replace' flag.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true. Description adds the key behavioral detail that the tool retrieves multilingual content, which is not in annotations. No contradictions. Acceptable disclosure for a simple read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb 'Get', no unnecessary words. Highly efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with two well-described parameters and no output schema, the description is complete. It covers the essential behavior and the specific 'multilingual' aspect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema provides 100% coverage with clear descriptions for libraryId and messageId. Description does not add additional parameter-level meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool retrieves the complete multilingual content of a library message, distinguishing it from sibling tools like list_library_messages (which lists messages without full content) and create/update/delete operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like list_library_messages. Usage is implied (get specific message by ID) but no when-not-to-use or alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description adds that it strips server-generated fields, making the tempering of the response transparent. It also explains the output is ready for create_question, providing some behavioral context beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler. The first sentence states the core purpose and output usage, the second provides a best-practice scenario. Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is simple (2 required params, no output schema, no nested objects), the description fully covers what the tool does, what it returns, and how to use the result. It also provides a concrete example, making it complete for agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for surveyId and questionId. The description does not add additional parameter meaning beyond what the schema provides, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets a question definition stripped of server-generated fields, specifically for reuse as a create_question template. It also gives a concrete use case (cloning complex question types), distinguishing it from get_question which returns raw definition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the best use case (replicating complex question types via UI then API) and how to use the output (pass via additionalFields). It does not explicitly state when not to use it or name alternatives, but the context is clear enough for the agent to infer appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly discloses beyond annotations that this GET can update contact-frequency state and reset email-status dates, providing critical behavioral context that annotations (readOnlyHint=false, destructiveHint=false) only hint at.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, and a critical warning. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While parameters are well documented in schema, the description does not elaborate on the output structure (e.g., format of links, pagination details). For a tool without output schema, more context would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameters described, so the description adds no extra meaning beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List one page of generated individual survey links', specifying the verb (list), resource (individual survey links), and pagination (one page). It distinguishes from siblings like list_distributions which list distributions, not links.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description warns of side effects and required scope, implying caution and suggesting when not to use (if avoiding state changes), but does not explicitly name alternative tools like list_distributions for read-only distribution listing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations only having destructiveHint=false, the description adds context about the tool's limitations (cannot rewrite answers) and that it updates only embedded data. However, it does not disclose job behavior, error conditions, or the nature of the operation beyond being a 'job'. Moderate transparency beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no waste: first sentence defines purpose and resource, second sentence clarifies limitations and alternatives. Perfectly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the primary function and limitations but omits details about the job mechanism (e.g., what the job returns, whether it's asynchronous) and does not mention the resetRecordedDate parameter. Given no output schema, some context is missing. Adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters well. The description adds minimal extra meaning, only clarifying that the tool updates embedded data (not answers), which is already implied. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool starts a job to update embedded data on an existing survey response. It uses specific verbs ('update embedded data') and specifies the resource ('survey response'), distinguishing it from sibling tools like create_response and response import.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states that this tool does not support rewriting answer values and directs users to use create_response or response import for that purpose. This provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds context beyond annotations: mentions BrandBaseURL and confirms no API write, consistent with readOnlyHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff, front-loaded with core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Sufficient for a simple read-only tool with one parameter and no output schema. Could note return format but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter with schema description coverage 100% (surveyId). Description adds no extra meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it looks up BrandBaseURL and returns anonymous survey URL. Distinguishes from siblings as the only tool for anonymous links without write operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (anonymous URL without distribution) and what it does not do (no write). Could mention alternatives like create_email_distribution for distributions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint: false, so the description doesn't need to reiterate non-destructiveness. It adds context about Matrix requirements (choices/answers) but does not disclose the important warning about literal `${` in questionJS (only in schema). Behavioral transparency is adequate but not enhanced beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no redundancy, front-loaded with main purpose. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (17 params, nested objects, no output schema), the description provides high-level context: full payload control, Matrix specifics, helpers, templates. It does not cover all parameter behaviors but is fairly complete for an overview.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description emphasizes the Matrix distinction, but the schema already explains choices and answers. No additional semantics beyond what schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a question in a survey block with full payload control. It distinguishes from sibling helpers by saying 'For common types, prefer the add_*_question helpers.' The verb and resource are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises when to use this tool versus alternatives: 'For common types, prefer the add_*_question helpers.' Also suggests using additionalFields for templates, giving clear context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations set readOnlyHint=false, which is consistent with the download action potentially saving files. The description adds behavioral details like automatic handling of large files and saving to a requested filename, providing useful context beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that convey the essential information with no redundant words. The purpose and prerequisite are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a download tool, the description explains the required fileId and mentions automatic handling of large files and optional filename. Without an output schema, it is sufficiently complete, though it could mention the response format (file stream or content).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description explains the role of fileId and saveToFile (custom filename) but does not elaborate on surveyId or format beyond schema. This adds marginal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool downloads a completed response export using a fileId from check_export_status. It distinguishes from sibling tools like export_responses and check_export_status by specifying the download action and prerequisite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use: after obtaining fileId from check_export_status. It implies a workflow but does not explicitly list when not to use or alternatives. Context is clear but exclusions are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond the readOnlyHint annotation by specifying what constitutes the 'complete definition'. It does not contradict the annotation and provides useful detail for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that is concise and front-loaded, with no extraneous information. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately lists the components of the definition. It does not cover error conditions or rate limits, but for a read-only retrieval tool, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not add meaning beyond the input schema, which already fully covers both parameters with descriptions. Baseline 3 is appropriate given 100% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a block's complete definition, listing specific components like question order, skip logic, randomization, loop-and-merge, and navigation options. This differentiates it from sibling tools like list_blocks, which likely provides only partial info.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for retrieving full block details, indirectly distinguishing from list_blocks. However, it lacks explicit 'when to use' or 'when not to use' guidance and does not mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description covers the core action (creation) but no further behavioral details beyond what annotations (destructiveHint=false) already convey. No mention of side effects, response details, or format specifics beyond the import format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no unnecessary text. Front-loaded with the core function and alternative recommendation. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters, no output schema, and minimal annotations, the description covers purpose and usage guidance well. Could mention return value (like survey ID), but not essential for selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds minimal detail about parameters: mentions 'base64-encoded DOCX' which maps to contentBase64, but does not elaborate on name or filename beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a survey from a base64-encoded DOCX' with a specific verb and resource. It distinguishes from import_survey_text by recommending that tool instead, differentiating its purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance: 'Prefer import_survey_text when possible because it is transparent and easier to validate.' Directly tells the agent when to use this tool vs. an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description describes a read operation ('List'). The description adds context about templates and discovery, which is useful beyond the annotation. It does not discuss pagination or response format, but with annotations covering safety, the description is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences with no fluff. It front-loads the action and resource, and each sentence adds value (action + usage context).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with readOnlyHint and 2 parameters, the description covers purpose and usage well. It could mention that results are paginated (implied by offset parameter) or describe the response structure, but given the lack of output schema, it is still informative enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters (libraryId and offset). The description does not add additional explanation or context for these parameters. Baseline score of 3 is appropriate as the description does not reduce ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'List' and the resource 'reusable survey questions in a Qualtrics library'. It distinguishes from sibling tools like list_questions by specifying the library scope, and adds context about using these as templates for discovery before copy/recreation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear use case: 'discover complex question definitions before copying or recreating them'. This guides when to use the tool. It does not explicitly exclude alternatives, but the sibling list implies other tools for direct creation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description confirms read-only behavior with 'List'. It adds pagination detail (nextPage) beyond annotations, which is useful behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the core purpose and a key detail (pagination). No redundant or unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core listing functionality and pagination hint, but lacks details on return format (e.g., survey objects) or how to use nextPage exactly. Given the low complexity and rich schema, it is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes both parameters with 100% coverage. The description adds minimal semantic value ('optional name filtering' matches filter parameter) but doesn't exceed what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List one Qualtrics-managed page of surveys' uses a specific verb and resource, clearly states the scope (one page), and distinguishes from sibling tools like create_survey or delete_survey by focusing on listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies pagination via 'use nextPage to continue', but does not explicitly state when to use this tool vs alternatives (e.g., search for surveys by name). However, context signals show many sibling tools are for create, update, delete, making the purpose clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotentHint=true and destructiveHint=false. The description adds context about the partial update mechanism (fetch-carry-modify) and warns about JavaScript escaping (avoid literal `${`), which goes beyond annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences plus a brief note about Matrix. All sentences are substantive, no redundant information. The main purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (13 parameters, nested objects) and no output schema, the description covers the key aspects: purpose, partial update, Matrix mapping, and a critical parameter warning. It is adequate for most use cases, though it could mention return value or permissions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter described. The description adds value by explaining that Matrix-specific fields (choices=rows, answers=columns) and providing crucial guidance on JavaScript escaping. This enhances understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing question' and specifies 'safe partial update' behavior, distinguishing it from sibling tools like create_question or delete_question. It also clarifies the relationship between choices and answers for Matrix questions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for modifying specific fields without sending the whole object, but it does not explicitly state when to use this tool versus alternatives like update_survey or when not to use it. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds important behavioral context beyond annotations, such as the default recursive merge and the effect of the replace parameter. Annotations already indicate idempotent and non-destructive, and the description aligns with them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the core purpose and then delivering critical nuance. Every word is meaningful, with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 25 parameters, no output schema, and nested objects, the description covers the key behavioral details (safe merge, replace option). It is mostly complete for decision-making, though it could mention return values or common error scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains each parameter. The description adds value by explaining the 'replace' parameter's role in the overall behavior, but does not elaborate on other parameters beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Safely patch survey options', using a specific verb and resource. It distinguishes itself from siblings like update_survey and get_survey_options by emphasizing the patch/merge behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to use the tool: by default it merges safely, and setting replace=true allows full replacement. It provides context for when to use each mode, though it does not explicitly compare to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool creates a version and can publish it immediately. Annotations only provide destructiveHint=false, so the description adds value by explaining the publishing behavior. However, it does not mention potential side effects (e.g., requiring a valid surveyId, effect on existing versions) or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the purpose, and contains no unnecessary words. Every sentence provides essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 4 parameters (1 required) and no output schema. The description covers creation behavior and the published flag but does not indicate what the response returns (e.g., version ID or object). Given the lack of output schema, this information would be helpful for an agent to handle the result. Also, no mention of prerequisite conditions or error scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds meaning by explaining that the 'version' parameter is a raw payload and that explicit arguments override matching keys, which is not obvious from the schema. It also clarifies the default for 'published'. This goes beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a survey version, distinguishes between draft and published via the 'published' parameter, and mentions that additional fields can be supplied. This verb+resource combination is unambiguous and differentiates from siblings like list_survey_versions or get_survey_version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that versions are drafts by default and that setting published=true immediately publishes, providing clear context on how to control behavior. It does not explicitly state when to use this tool versus alternatives, but no direct alternative for creating a version exists among siblings, so the guidance is adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotentHint: true and destructiveHint: true. The description adds that the entire tree is replaced and requires retrieving the current flow first, but does not disclose additional traits like permissions or what happens to existing elements not in the new tree.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: the first states the purpose, the second provides usage instructions. No redundant information, front-loaded with key action. Extremely concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description should ideally indicate what the tool returns (e.g., success confirmation). It does not. While the usage flow is clear, missing return value details and error conditions reduce completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with descriptions for both parameters. The description reinforces that 'flow' should be the full tree object, but does not add new semantic meaning beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Replace the entire survey flow tree,' specifying the action (replace) and resource (survey flow tree). It distinguishes from sibling tools like insert_flow_element and update_flow_element, which modify parts of the flow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to 'Use get_survey_flow first to get the current flow, modify it, then pass the full tree back.' This provides a clear workflow and indicates this tool is for complete replacements, not partial updates.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that large exports are automatically saved to the Downloads folder, a behavioral trait beyond what annotations provide (readOnlyHint=false). It does not cover other potential side effects or async behavior, but the main side effect is addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with a clear list of uses. No fluff, every sentence adds value. Front-loaded with the main purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 12 parameters and no output schema, the description covers the main filtering capabilities and the auto-save behavior. It omits details on format, labels, and other options, but the schema handles those. Reasonably complete for high-level understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the schema already documents parameters. The description adds value by grouping key parameters (date bounds, saved filter, question selection, in-progress) and recommending their use, which aids understanding beyond schema entries.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool exports survey responses with filters, and distinguishes from the sibling 'export_responses' by emphasizing 'documented Qualtrics filters' to reduce data size. The action and resource are explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use (to reduce data size with filters) and lists specific filter types. However, it does not explicitly compare to the unfiltered 'export_responses' sibling or state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only indicate non-destructive behavior. The description adds one behavioral constraint (source of QSF) but omits other details like expected response, error behavior, or idempotency. With limited annotations, more behavioral context would be helpful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The critical guidance (format source) is front-loaded, and every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description is adequate for selection. It clarifies input format and constraints. However, it could mention what the tool returns (likely survey ID) or post-creation state to fully complete the usage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 3 parameters with descriptions. The description adds crucial context: the format for the 'qsf' parameter (must be from UI export, not version API), which goes beyond the schema. No enums or nested parameter details are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a survey from a QSF object, specifying the exact format and API used. It distinguishes from sibling import tools (import_survey_text, import_survey_from_url, import_survey_docx) by naming the specific QSF format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool (with QSF exported by Qualtrics UI) and warns against using QSF from the Survey Version API, providing clear alternatives and avoiding misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true; the description adds that the tool returns directoryId in the format 'POOL_...' and that it is needed for other tools. This goes beyond annotations by hinting at the output structure, though it does not detail pagination behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two short, front-loaded sentences that deliver purpose and usage guidance with no extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (2 optional parameters, fully described schema, read-only annotation), the description is nearly complete. It explains the purpose, output value, and usage context. Minor omission: no mention of pagination in the description, but the schema covers it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters (pageSize and skipToken). The description adds no additional meaning beyond what the schema provides, hence a baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List the XM Directories (contact pools) available to this account', specifying the verb (list) and resource (directories/contact pools). It also distinguishes this tool from siblings by highlighting that it is used to discover the directoryId required by other mailing-list and contact tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to 'Use this first to discover the directoryId... that every mailing-list and contact tool requires.' This provides a clear when-to-use directive, indicating this tool is a prerequisite for other operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare idempotentHint=true and destructiveHint=false, indicating safe repeated use with no destructive side effects. The description adds context by grouping scopes by risk level and noting that an empty array sets fully read-only, which clarifies behavior beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences that efficiently state the action, list scopes with risk levels, and provide a usage principle. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple configuration tool with one parameter and no output schema, the description covers the essential context: what the tool does, which scopes are available, and how to achieve read-only mode. It is missing a note on return values, but the simplicity mitigates this gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the parameter fully (100%), but the description adds value by categorizing scopes by risk and explaining the effect of an empty array. This helps the agent choose appropriate values beyond the raw enum.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool enables or disables write permissions for specific categories, and it lists categories grouped by risk. This is a specific verb+resource combination that distinguishes it from siblings like set_read_only_mode.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises using the tool to grant minimum permissions, which implies when to use it. However, it does not explicitly mention when not to use it or provide direct comparisons to alternatives like set_read_only_mode, though the risk categorization aids in selecting scopes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond the annotations (idempotentHint, destructiveHint). It explains the fetch-and-merge pattern required by Qualtrics' PUT semantics, which is critical for correct invocation. It also declares the operation as 'safe', aligning with destructiveHint=false. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that front-load the purpose and then provide essential behavioral detail. Every sentence adds value. No filler or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, nested objects, no output schema), the description covers the key behavioral nuance (fetch-and-merge) and lists supported features. It does not explain return values or error conditions, but the lack of output schema reduces the burden. It is sufficient for an experienced agent familiar with block definitions, though a note about prerequisites (e.g., block must exist) could improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameters are already well-documented. The description mentions 'question ordering, skip logic, randomization, and loop-and-merge,' which map to the 'options' and 'blockElements' parameters, but does not add new meaning beyond what the schema provides. For a tool with full schema coverage, baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Safely update part of a block definition.' It lists specific capabilities (question ordering, skip logic, randomization, loop-and-merge) that distinguish it from other block tools like create_block or delete_block. The verb 'update' combined with 'part of a block definition' is precise and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the update mechanism ('fetched and carried forward'), which helps the agent understand how to use it. However, it does not explicitly contrast with alternatives such as full replacement via a different tool or when to prefer update_block over create_block. The context of sibling tools provides some implicit guidance, but explicit when-not-to-use instructions are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotent and non-destructive. Description adds key behavior: existing content is fetched and preserved unless explicitly replaced, which implies merge/replace semantics and safety. Does not cover error conditions but adds value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with verb and resource, then behavioral nuance. No wasted words; every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity and presence of annotations, the description covers the main behavioral nuance (merge/replace) and safety. It does not explain return values or require existence, but that is acceptable for an update tool without output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions. The description clarifies that the 'messages' parameter merges with existing content and 'description' parameter replaces, adding semantic value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The tool name 'update_library_message' and description 'Safely update a library message' clearly specify the verb (update) and resource (library message). It distinguishes from siblings like create, get, delete, and list by implying partial update with merge behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies use when you want to update a library message while preserving existing content unless explicitly replaced. Does not explicitly state when not to use or compare to alternatives, but sibling tools are distinct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (idempotentHint, destructiveHint), the description reveals the merge mechanism and the effect of replace=true. It adds valuable behavioral context and confirms non-destructive, idempotent nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three efficient sentences: purpose, default behavior, and usage caveat for replace. Every sentence adds value, front-loaded and free of fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers key aspects: purpose, parameter behavior, permissions. Without an output schema, it does not detail return values, but it is sufficient for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the schema already documents parameters. The description enhances understanding by explaining the default merge behavior and the role of replace=true, adding meaning to the translations parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update translations for one language' with specific action and resource. It distinguishes from siblings like get_survey_translations and explains the merge vs replace behavior, leaving no ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides specific guidance on when to use replace=true and mentions required permissions. While it does not explicitly contrast with alternative tools, the name and context make usage clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide idempotentHint and destructiveHint. The description adds behavioral context: it blocks all writes when enabled and enables all write scopes when disabled. This aligns with annotations and adds useful nuance, though it doesn't detail auth requirements or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, and no extraneous words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one boolean parameter, no output schema, and comprehensive annotations, the description provides all necessary context: what it does, when to use it, and an alternative. Nothing missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'enabled' has a clear description in the schema (100% coverage). The tool description repeats this same meaning without adding extra nuance, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool toggles read-only mode on/off, blocking or enabling all writes. It uses a specific verb+resource ('toggle read-only mode') and distinguishes itself from 'set_write_scopes' which offers finer control.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Quick toggle' for a fast mode switch and advises using 'set_write_scopes instead' when finer control is needed, providing clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, and the description reinforces this with 'Read-only preflight.' It adds detailed behavioral insight into the specific checks performed, which goes beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, and no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter tool with no output schema, the description fully covers what the tool does, its read-only nature, and the specific validations performed. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter with a description in the schema itself (100% coverage). The tool description does not add new parameter details, which is acceptable given the schema already describes it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a 'Read-only preflight' for survey validation, listing specific checks (FlowID uniqueness, block references, etc.). It distinguishes itself from sibling tools like activate_survey or update_survey by focusing on validation before activation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises using the tool 'before activation or publishing,' providing clear context. However, it does not explicitly mention when not to use it or offer alternatives, though the purpose is well understood.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors beyond annotations: read-only mode for GET, scope requirements for writes, and that known endpoints retain normal scope. No contradiction with destructiveHint annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each purposeful: purpose, behavior, guidelines. No fluff, front-loaded with critical information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks description of return value (no output schema), but as a generic API tool, return format is implicit. Covers safety, scope, and usage context well.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions (100% coverage). Description adds no parameter-specific details beyond what schema provides, achieving baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly defines the tool as an escape hatch for undocumented Qualtrics API v3 endpoints, specifying the resource (API v3) and verb (requests). It distinguishes from siblings by requiring endpoints not covered by dedicated tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (endpoints without dedicated tools) and when to avoid (prefer dedicated tools for validation and safer defaults). Also distinguishes read-only GET vs. write operations requiring advanced scope.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes key behavioral traits beyond annotations: auto-saving large exports to local file to avoid context limits, and that small exports are returned directly. Does not cover potential performance implications or exact file location details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences. First sentence states purpose, second highlights critical auto-save behavior with emphasis, third provides alternative. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main functionality and guidance. Lacks explicit description of return values for small exports (direct response format), but given no output schema, this is acceptable. Provides enough for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds meaning beyond schema by explaining the auto-save behavior, recommending 'saveToFile', and noting the format options. Schema has 100% coverage, but description provides extra context and usage advice.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Export survey responses in JSON or CSV format', specifying verb and resource. Differentiates from sibling 'export_responses_filtered' by mentioning it for better control over data size.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends 'export_responses_filtered' for filtered exports, explains auto-save behavior for large exports, and advises using the 'saveToFile' parameter for easier access.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide idempotentHint and destructiveHint. The description adds context about PUT semantics ('replaces omitted fields'), which is valuable beyond the annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Efficiently conveys purpose, usage, and critical behavioral note. Front-loaded with the key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with a nested object parameter and a note about PUT semantics, the description is complete. It covers how to construct the payload, the need for a prior GET, and the risk of omitting fields. No output schema needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds value by explaining the workflow: use get_quota first and modify the returned definition, clarifying the 'complete payload' requirement for the quota parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'replace' and the resource 'quota with a complete raw Qualtrics definition'. It differentiates from siblings like get_quota, create_quota, and delete_quota by emphasizing full replacement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to retrieve the quota first via get_quota, then modify and send the complete payload, warning about PUT behavior replacing omitted fields. Provides clear when-to-use and why.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate idempotentHint true and destructiveHint false. The description adds that PUT replaces omitted fields, which is consistent and provides extra context about the tool's behavior beyond what annotations convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no wasted words. The purpose is front-loaded in the first sentence, and the second provides essential workflow guidance. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 parameters, all required, and an object parameter that lacks full schema constraints, the description properly advises fetching the current definition to get the structure. No output schema exists, but the workflow guidance compensates. The tool is completely described for the intended use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds meaning: it specifies the 'quotaGroup' parameter must be a 'complete raw Qualtrics definition' and advises first retrieving the existing group. This helps the agent understand the parameter's content beyond the schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Replace a quota group with a complete raw Qualtrics definition,' using a specific verb and resource. It distinguishes from sibling tools like create_quota_group and update_quota by emphasizing replacement with a full definition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Use get_quota_group first, modify the returned definition, and send the complete payload because PUT may replace omitted fields.' This tells the agent when to use the tool and how to prepare the input.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds important behavioral details beyond annotations: explains that the complete current options resource is fetched and preserved, and that keys are merged by default. No contradiction with idempotent and non-destructive hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three succinct sentences covering purpose, mechanism, behavior, and permission. No redundancy or unnecessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Comprehensive for a mutation tool, though missing explicit return value (output schema absent). Still sufficiently complete for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds value to the already well-described schema by explaining the availableLanguages parameter as a 'patch keyed by language code' and clarifying the replace parameter's effect on merging vs. replacement.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates the AvailableLanguages option map via PUT, specifying the mechanism (fetches and preserves current options) and distinguishing from siblings like get_survey_languages or update_survey.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions required permission ('Requires Translate Surveys permission') and describes default merge behavior with option to replace, but could improve by stating when not to use or listing alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes beyond the destructiveHint annotation by detailing cascading deletion behavior and the need for explicit confirmation, which annotations alone do not capture.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first clearly states action, second adds essential usage guidance. No unnecessary words, front-loaded with core function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a delete tool without output schema, the description covers destructive nature, cascade effect, required confirmations, and references documentation. No gaps for the agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions (100% coverage), so baseline is 3. The description adds value by explaining why confirmCascade is necessary (cascade behavior), reinforcing semantics without redundancy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource ('Permanently delete a quota group AND every quota in that group'), clearly distinguishing it from sibling tools like delete_quota which only deletes a single quota.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states to inspect membership first and acknowledge the cascade, providing clear when-to-use and when-not-to-use context without needing to name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![qualtrics-mcp-server MCP server](https://glama.ai/mcp/servers/yrvelez/qualtrics-mcp-server/badges/card.svg)](https://glama.ai/mcp/servers/yrvelez/qualtrics-mcp-server)

Score Badge

Copy to your README.md:

[![qualtrics-mcp-server MCP server](https://glama.ai/mcp/servers/yrvelez/qualtrics-mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/yrvelez/qualtrics-mcp-server)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/yrvelez/qualtrics-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server