SAS MCP Server

Official

by sassoftware

Overview Schema Related Servers Score Discussions

Python

Remote

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.11.1
Disambiguation5/5
Each tool targets a distinct resource-action within clear domains (jobs, CAS data, compute, files, reports, catalog, business rules, decision flows, ML). Despite 75 tools, there is no overlap; e.g., get_job_status vs get_job_log, list_castables vs list_compute_tables are clearly separated.
Naming Consistency5/5
All tools use a consistent snake_case verb_noun pattern (e.g., list_caslibs, get_castable_columns, create_decision_flow, delete_report). No mixing of styles like camelCase or inconsistent verb choices.
Tool Count2/5
75 tools far exceeds the 25+ threshold for 'too many' per the rubric. While the breadth of SAS Viya services justifies a large surface, this count is excessive for an MCP server and increases cognitive load.
Completeness4/5
The tool set provides comprehensive lifecycle coverage across major SAS Viya services: job execution, compute sessions, CAS data, files, reports, business rules, decision flows, ML projects, and catalog. Minor gaps like missing file deletion or CAS table removal exist, but core workflows are well-covered.
Average 4.1/5 across 74 of 75 tools scored. Lowest: 2.7/5.
See the Tool Scores section below for per-tool breakdowns.
- 5 of 5 community issues answered or closed in the last 6 months
- 92 commits in the last 12 weeks
- Last stable release on August 20, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under Apache 2.0.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds only a trivial scope ('Viya environment') and does not mention pagination behavior implied by limit/start parameters or filtering via filter_name. Since annotations carry the main behavioral disclosure, a 3 is appropriate; the description does not contradict them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words, front-loading the action and resource. However, it is likely under-specified for the given complex tool, but conciseness itself is efficient. The lack of additional context is more a completeness issue, so this dimension gets a 4.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema and thorough annotations, the description lacks critical context: no explanation of what a compute context is, no usage guidance, no parameter semantics, and no mention of pagination or filtering. The tool is simple, but an agent cannot determine when to invoke it or how to use the optional parameters without more detail. This is insufficient for accurate tool selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for the three parameters (limit, start, filter_name). It provides no meaning beyond the schema's default values, making it impossible for an agent to understand the semantics of filter_name or pagination parameters. This is a significant omission, warranting a score of 1.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'compute contexts' with an environment scope ('Viya'). It distinguishes from sibling tools like list_compute_libraries or list_compute_tables because the resource is different, though it doesn't explicitly explain what a compute context is or how it differs from those siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance is given about when to use this tool versus alternatives. The description simply states the action without any context about typical use cases, prerequisites, or exclusions. There is no mention of alternative tools for listing different compute-related resources.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare destructiveHint=true and idempotentHint=true, so the description is not required to state those. However, it adds no additional behavioral context, such as what happens if the job is not running, whether cancellation is reversible, or any side effects. It merely repeats the tool's purpose without enriching the user's understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one short sentence), but it provides little information beyond the tool's name. While it is front-loaded and has no wasted words, it lacks any substantive elaboration that would make it more useful to an agent. It is appropriately short for a simple tool but is under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter, full schema coverage, an output schema, and annotations covering safety (destructive, idempotent), the description is adequate but minimal. It does not address edge cases like attempting to cancel a non-running or already-completed job, which could be important for an agent. Overall, it is sufficient for a basic cancel operation but lacks completeness in explaining behavior under all conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100% for the only parameter (job_id), which is clearly documented as 'ID of the job to cancel.' The description itself adds no extra meaning beyond what the schema already provides, so it meets the baseline but does not exceed it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action ('Cancel') on a specific resource ('a running job'). It is unambiguous and differentiates from sibling tools like 'get_job_status' or 'list_jobs'. However, it does not specify the scope or types of jobs, which keeps it from being a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no mention of prerequisites, and no exclusions (e.g., cannot cancel completed jobs). It simply states the action without any contextual usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations only indicate that the tool is not read-only, idempotent, or destructive; the description adds no additional behavioral context. It does not disclose whether running a project is asynchronous, whether it creates a job, or what side effects it may have on the project.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no filler or repetition. It is concise, though slightly too terse to provide useful operational context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists, the description omits important context such as whether the run is asynchronous, whether it submits a job, and how it relates to get_job_status or cancel_job. For a tool that likely triggers a pipeline, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema fully documents the single parameter project_id as 'ID of the project to run,' so the description adds no extra meaning beyond the schema. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Run') and resource ('AutoML pipeline automation project'), which distinguishes it from create_ml_project and list_ml_projects. However, it does not clarify what 'run' entails, such as whether it starts an asynchronous job or executes synchronously.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives like submit_batch_job, execute_sas_code, or create_ml_project. It also does not mention prerequisites or follow-up actions such as checking job status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no additional behavioral details beyond the basic listing function, but it does not contradict the annotations; it remains neutral with minimal added value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence with no redundant words. It is appropriately concise and front-loaded, providing the essential purpose without any fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is an output schema and the input schema is fully documented, the description does not need to explain return values or parameters. However, the tool is part of a large sibling group and the lack of differentiation or usage context leaves the description minimally adequate but not complete for an agent to fully understand its role relative to other tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters ('limit' and 'filter_name') well-described in the schema. The description adds no extra meaning beyond what the schema already provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List Visual Analytics reports' with a clear verb and resource. However, it does not differentiate from siblings like 'get_report' or other list tools; the tool name itself is clear but the description lacks additional context on what distinct functionality or scope it covers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as 'list_jobs' or 'list_ml_projects'. The description implies it is for listing reports, but does not mention any context, prerequisites, or exclusions, offering no usage guidance beyond the obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds no behavioral details beyond the basic 'check' action, such as what statuses might be returned or how it handles non-existent job IDs. It does not contradict annotations, but adds minimal value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no unnecessary words. It is front-loaded with the key action and resource, making it quick to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple getter with an output schema present and annotations covering safety, the description is minimal but adequate. However, given the presence of sibling tools like get_job_log and list_jobs, a bit more context on return values or status categories could improve completeness. It is neither under-specified nor over-specified for a basic tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for job_id, and the parameter is simple. The description does not add any extra meaning (e.g., how to obtain the job_id, format, or relationship to submit_batch_job output). It meets the baseline but does not go beyond it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (check) and the resource (status of a submitted job). It distinguishes from siblings like list_jobs (listing all jobs) and get_job_log (retrieving logs), but does not elaborate on the nature of the status or return values, so it is clear but not fully distinctive.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description does not mention when to call it (e.g., after submission, to poll progress) or when to prefer list_jobs or get_job_log. Usage context is entirely absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only, idempotent, and non-destructive. The description's use of 'Fetch' is consistent with this, but it adds no extra behavioral details beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words or fluff. It directly states the tool's action and target.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description tells what the tool does but does not explain the meaning of 'locked' or any prerequisites, nor does it clarify what 'content' includes. Given the output schema exists, return details are not needed, but the description could be more complete about the nature of the revision.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already fully describes both parameters (decision_id and revision_id) with clear explanations. The description does not add any additional meaning or context for these parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches the content of a specific locked decision revision, distinguishing it from sibling tools like get_decision_flow that retrieve the flow itself or list_decision_flow_revisions that list revisions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no information about when to use this tool versus alternatives, such as when a locked revision is needed or how it differs from other retrieval functions. No usage context is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description is not required to repeat these. The description 'List models' is consistent with a safe, idempotent read operation. It adds no extra behavioral context (e.g., about pagination or result ordering), but given the strong annotation coverage, the minimal description meets the baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It is appropriately sized for a simple list operation and front-loads the action and resource. No unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, a well-documented parameter schema, and comprehensive annotations, the description is adequate but not rich. It does not clarify what constitutes a 'registered model' versus other model states (e.g., champion models) or whether any implicit scoping exists. However, for a simple list operation, the essentials are covered, so it is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is only one parameter (limit) and the schema fully describes it with default and meaning (100% coverage). The tool description adds nothing beyond the schema, so the baseline score of 3 applies. It neither clarifies nor obscures the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List models in the Model Repository' clearly states the action (list) and the resource (models in the Model Repository). It is specific enough to distinguish from sibling tools like list_jobs or list_ml_projects by naming the resource type. However, it does not explicitly differentiate it from other model-related listing tools, but the resource name is sufficient.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as other listing tools or model-specific actions like register_ml_champion_model. There is no mention of prerequisites, filters, or typical use cases. Without any contextual hints, an agent may not know when this is the appropriate choice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds that it returns 'metadata and definition', which gives a bit more insight into the response content, but it does not disclose any additional behaviors such as error handling, permissions, or output details. Since annotations handle safety, the description adds limited but non-trivial context, so a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, compact sentence that immediately captures the core function. There is zero redundancy, and it is front-loaded with the action verb. Every word contributes meaning, making it an exemplary concise description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter), the presence of an output schema, and the strong annotations, the description is complete enough for an agent to understand what the tool does. It clearly states the purpose and relies on schema/annotations for technical details. It does not discuss edge cases or error behavior, but for a read-only getter with output schema, this is sufficient. A 4 is appropriate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description covers 100% of the only parameter (report_id) with 'ID of the report.' The tool description does not add any additional semantics for the parameter, relying entirely on the schema. Given the high coverage, a baseline of 3 is warranted, and the description does not go beyond that.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the verb 'Get' and the resource 'a Visual Analytics report's metadata and definition', which clearly identifies the tool's action and target. While it doesn't explicitly differentiate from siblings like 'get_report_outline' or 'describe_report_objects', the phrase 'metadata and definition' adds specificity that helps distinguish it. It is clear, but not strongly contrastive with alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives. It simply states what it does without mentioning conditions, prerequisites, or exclusions. There is no reference to sibling tools or use cases where another tool would be more appropriate, leaving the agent without explicit usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is known. The description adds no extra behavioral context (e.g., pagination, sorting, full response details) beyond the bare action, offering little value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no filler. It is immediately clear and efficiently conveys the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, return values need not be explained. The tool has only two optional parameters and the annotations cover safety, so this simple description is adequate, though it could mention the default limit or filter semantics for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters (limit and filter_name) fully described in the schema. The tool description adds no extra meaning about parameters, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'files in the Viya Files Service', which distinguishes it from sibling tools for other resources (e.g., list_jobs, list_reports). It is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives, no exclusions or context about file-specific scenarios. The description simply states what it does without any usage instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safe read-only profile is covered. The description adds only the notion of 'available' destinations, but does not disclose pagination behavior, return shape, or other behavioral details beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, 'List available publishing destinations,' which is directly front-loaded and contains no filler. It is appropriately sized for the simple read-only operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three optional pagination/filter parameters, strong annotations, and an output schema, the one-line purpose is mostly sufficient for agent selection and invocation. However, it does not clarify what makes a destination 'available' or provide decision context relative to other publishing/list tools, leaving minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%: limit, start, and filter_name each have descriptions, so the schema carries the parameter semantics. The description contributes no additional parameter detail and does not explain how 'available' interacts with filter_name, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and a specific resource ('publishing destinations'), clearly stating the operation. It is distinct from sibling list tools by subject matter, though it does not elaborate on the meaning of 'available' or distinguish against a closely related tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to invoke this tool instead of other list tools (e.g., list_reports, list_jobs) or whether pagination/filtering should be preferred. The use case is only implied by the tool name and resource, so the agent receives no explicit decision criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations include destructiveHint: true, so the description must add context about destructive behavior. The description does not mention any side effects, permissions, or irreversibility. It only states the action, which is not misleading but misses an opportunity to elaborate on what 'publish' entails beyond the annotation. This is a moderate gap given the destructive hint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, no fluff, front-loads the verb 'Publish' and the object. Every word is meaningful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has a destructive annotation and an output schema (which likely describes the published model details), the description is minimally viable. It tells the agent what it does, but lacks details on preconditions (e.g., that the champion model exists), or potential failures. The output schema covers return values, so that's not an issue, but the description could be more helpful with context about the pipeline automation project requirement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and both parameters have clear descriptions. The description adds no further semantic detail beyond 'project_id' and 'destination_name', so it does not elevate beyond the baseline. The name 'destination_name' is somewhat ambiguous, but the schema clarifies it as 'Name of the destination to publish to.' No additional clarification is offered.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Publish the champion model') and the resource ('from an AutoML pipeline automation project to the Model Repository'). It distinguishes from sibling tools like register_ml_champion_model and publish_decision_flow, though it doesn't explicitly differentiate between those, but the specific 'champion model' and 'Model Repository' make the purpose clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. The context implies it is used after an AutoML project has a champion model, likely after run_ml_project. It doesn't mention alternatives or prerequisites, but the purpose is clear enough for an agent to infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows it is a safe read operation. The description adds little beyond 'fetch rows', but it does imply a read-only behavior. Given annotations carry the safety profile, the description adds minimal extra behavioral context, so a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no filler. It front-loads the purpose and includes a useful detail about column names. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists and annotations are present, the description is sufficient for a simple data retrieval tool. It could mention pagination or row limits, but the schema already covers that. The tool is straightforward, so the description is adequately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, meaning parameters are well-described in the schema (e.g., limit, start, server_id, table_name, caslib_name). The description adds no additional parameter meaning beyond 'with column names' which is not directly tied to a parameter. Since schema covers parameters fully, baseline 3 is set.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Fetch rows from a CAS table with column names' clearly states the action (fetch rows) and resource (a CAS table), and hints at the result including column names. While it doesn't explicitly differentiate from sibling tools, the name and description align with table data retrieval, distinguishing it from list_castables or get_castable_columns which list metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for retrieving table data but does not explicitly state when to use it over siblings like list_castables, get_castable_columns, or query_data. No guidance on use cases or exclusions is provided. Since the name and description are clear, the usage context is implied but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. However, the description adds no extra behavioral context such as pagination behavior, default limit interaction, or error handling. Given the annotations, this is acceptable but not additive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no filler. It front-loads the primary action and resource, achieving maximum clarity with minimal verbiage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a straightforward listing tool, the description is sufficient. The output schema and parameter descriptions cover return values and parameter details. However, it could optionally mention that it is a read-only operation or that the limit parameter controls response size, though these are already covered by annotations and schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides full descriptions for both parameters (server_id and limit), covering 100% of parameter semantics. The tool description adds nothing beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('CAS libraries (caslibs)') with a specific scope ('available on a CAS server'). It distinguishes from sibling tools like list_compute_libraries or list_castables by focusing on caslibs, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to choose this tool over alternatives such as list_cas_servers or list_compute_libraries. The description implies usage for listing caslibs but does not mention exclusions, alternate tools, or prerequisite conditions like server availability.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description is consistent with a read-only listing operation. The description adds only the CAS-library scope and no additional behavioral detail such as pagination, permission requirements, or whether only in-memory tables are returned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler or redundant information. Every word contributes to stating the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present and annotations covering the safety profile, the description is mostly sufficient for a simple listing tool. However, it leaves some contextual ambiguity around how this tool relates to sibling list tools such as list_compute_tables and list_source_tables.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully documents all three parameters (server_id, caslib_name, limit) with 100% coverage, so the description does not need to compensate. The description adds no parameter-level meaning beyond what the schema already provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('tables') scoped to 'a CAS library,' making the core operation clear. However, it does not distinguish this from sibling tools like list_compute_tables or list_source_tables, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied: use this tool to list tables in a CAS library. However, there is no explicit guidance about when to choose this over alternatives such as list_compute_tables or list_source_tables, and no exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only, idempotent, and non-destructive behavior, and the description does not contradict them. It adds the 'locked revisions' scope, but it makes a broad claim of 'all' without clarifying pagination or the limit default of 20, which is a behavioral gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler or repeated schema information. Every word contributes to the meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, safe, and backed by a complete schema and output schema, so the one-line description is mostly sufficient. The main remaining gap is that 'all' is not fully aligned with the limit parameter, and no pagination behavior is mentioned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with decision_id and limit already well-described in the input schema. The tool description adds no extra parameter-level meaning beyond what the schema provides, so this is at the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all locked revisions of a decision flow' uses a specific verb and resource, and the 'locked revisions' qualifier distinguishes this from listing decision flows or getting a single revision. It is clear and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no explicit 'when to use' or alternative-tool guidance, such as 'use get_decision_flow_revision for a specific revision' or 'use lock_decision_flow_revision to create a locked revision.' The use case is only implied by the tool's name and phrasing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a write operation (readOnlyHint=false) and non-destructive (destructiveHint=false), but the description does not add behavioral context such as whether existing models are overwritten, whether permissions are needed, or what happens if the champion model doesn't exist. With annotations present, the bar is lower, but the description adds minimal beyond the basic action. It is not misleading and does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that conveys the essential information without any fluff. It is front-loaded and every word adds value, making it efficient for an agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter, and the description adequately conveys the core function. Since an output schema exists (though not shown), return values are documented elsewhere. The description is complete for the action, though it could have mentioned the need for a champion model in the project, but that is a minor omission given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter project_id is fully described in the schema (coverage 100%). The description does not elaborate on the parameter beyond its schema description, nor does it clarify any nuances like required project state or relationship to the champion model. Since schema coverage is complete, a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Register the champion model' and specifies the source ('from an AutoML pipeline automation project') and destination ('to the Model Repository'). It uses a specific verb and resource, making the tool's purpose unambiguous. While it doesn't explicitly contrast with sibling 'publish_ml_champion_model', the distinct action 'register' is clear enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like publish_ml_champion_model or other registration/publishing tools. It does not mention prerequisites, scenarios, or exclusions. The user is left to infer when to register vs. publish, which is insufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are all false, so they do not convey a clear safety or side-effect profile. The description does not disclose whether scoring triggers a remote execution, creates a job, or produces side effects. 'Score data' implies an inference operation, but with readOnlyHint=false the agent cannot confidently assume no state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler. It states the action, target, and domain efficiently, earning every word.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although the parameter schema is complete and an output schema exists, the tool description is too thin for a compute-oriented operation. It does not mention how to discover required module_id and step_id values via sibling tools like list_mas_modules or get_mas_module_step_signature, nor does it describe execution semantics or potential side effects. The description leaves important workflow context implicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all three parameters, providing a baseline of 3. The description adds useful semantic framing by specifying that module_id refers to a published model or decision and that input_data is the data being scored, which goes beyond the schema's generic 'MAS module ID' wording.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('score') with a clear resource ('data against a published model or decision (MAS module)'). It distinguishes this tool from siblings like list_mas_modules and get_mas_module_step_signature by stating the actual scoring action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used when data needs to be scored against a published MAS model or decision, but it does not provide explicit when-to-use/when-not-to-use guidance or reference alternatives. The 'MAS module' qualifier gives some context, but the agent must infer prerequisites like module_id and step_id discovery.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, idempotentHint=true, and destructiveHint=false, which fully cover the safety profile. The description adds the 'current state' context, but does not disclose any behavioral traits like response format or pagination. With annotations handling the safety, the description adds some value but not substantial behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and to the point, with no wasted words. It front-loads the purpose and is appropriately sized for a simple fetch operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (1 parameter, output schema present, annotations comprehensive), the description is reasonably complete. It says what the tool does and the schema covers parameters. The output schema exists, so return values are covered. The only minor gap is lack of explicit guidance on usage scenarios, but overall it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage; the parameter 'decision_id' is described as 'The decision flow UUID (not its name — list_decision_flows returns both).' This adds meaning beyond the schema by clarifying the format and providing a cross-reference to another tool. The description mentions 'flow' but not the parameter explicitly, so the schema does the heavy lifting, yet the additional note about UUID vs name is helpful.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the function: 'Fetch the current state of a SAS Intelligent Decisioning flow.' This is a specific verb ('fetch') and resource ('decision flow'). It distinguishes from siblings like 'get_decision_flow_code' and 'list_decision_flows' by focusing on 'current state', but doesn't explicitly compare to alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit when-to-use or when-not-to-use guidance. It implies that it's used to retrieve a flow's current state, but doesn't contrast with related tools like 'get_decision_flow_revision' or 'list_decision_flows'. The sibling list offers a clear context, but the description itself lacks direct guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the operation readOnly, idempotent, and non-destructive. The description adds the scope ('inside a rule set') but includes no behavioral details such as pagination, result ordering, or the meaning of 'all' relative to the limit parameter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the key action and resource, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple and has output schema plus strong annotations, but the phrase 'all rules' is undermined by the limit parameter (default 100) and no mention of pagination. This is a meaningful gap for a list operation, preventing a higher score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both ruleset_id and limit already described. The description does not add further parameter semantics beyond what the schema provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists rules within a SAS Business Rules rule set, using a specific verb and resource. It distinguishes itself from siblings like list_business_rulesets (lists rule sets) and get_business_rule (retrieves a single rule).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended usage is implied: call with a ruleset_id to enumerate its rules. However, there is no explicit guidance on when to choose this over alternatives like get_business_rule or list_business_rulesets, nor any exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds the 'recent' scope and service source, but leaves 'recent' undefined and does not mention ordering or pagination behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded with the verb and resource, contains no filler, and is appropriately sized for a simple list tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter list tool with an output schema and strong annotations, the description is mostly adequate. However, 'recent' is vague, and there is no mention of whether jobs are scoped to the current user or workspace, or how results are ordered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single 'limit' parameter is already well-described in the input schema. The description adds no additional parameter semantics beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') with a clear resource ('jobs') and service context ('Job Execution service'). It distinguishes itself from sibling tools like get_job_status and get_job_log by being the collection-listing operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a list of recent jobs is needed, but it does not explicitly state when to prefer this tool over alternatives such as get_job_status or get_job_log. No exclusions or alternative tool references are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows it is safe. The description adds no extra behavioral context beyond the basic listing, but it is consistent with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that is front-loaded with the action and resource. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional parameter, an output schema, and comprehensive safety annotations, the description is complete enough. No additional context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter 'limit' that has a clear description and default. The description does not add any additional parameter semantics beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the specific resource 'AutoML pipeline automation projects', which distinguishes it from other list tools like list_jobs and list_decision_flows.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, and no exclusion or context is given beyond the basic listing operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds the important behavioral label 'Permanently delete,' reinforcing irreversible destruction beyond the annotations' destructiveHint=true. It does not elaborate on effects on published/locked revisions, but the core destructive outcome is clearly disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one short, front-loaded sentence with no filler or redundancy. Every word contributes to the core purpose and consequence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter destructive tool with strong annotations and an output schema, the description sufficiently communicates the critical information. Minor additional context about what happens to published or locked variants could be useful, but the core deletion use case is realistically complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides full 100% coverage for the single parameter, including the meaning 'UUID to delete' and the clarification that it is not the name. The tool description itself adds no parameter semantics beyond what the schema already supplies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('delete') and the specific resource ('a SAS Intelligent Decisioning flow'), and it is distinguishable from sibling delete tools targeting other resources. It stops short of a 5 because it does not explicitly differentiate this operation from related decision-flow operations such as publishing or locking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended usage is implied by the action 'delete' but the description gives no explicit when-to-use or when-not-to-use guidance, nor does it name alternatives. The schema's mention that list_decision_flows can provide the UUID is useful, but it is operational detail rather than usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description does not need to restate safety. It adds context that the returned artifact is generated DS2 execution code, but does not disclose additional behavioral traits such as whether the code depends on a published revision or whether retrieval can fail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler or redundant information. It states the exact action and resource economically, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, read-only, idempotent retrieval tool with one well-documented parameter and an output schema available, the description is sufficiently complete. It clearly names the resource being retrieved and is consistent with annotations, though a note about how the generated code relates to decision flow revisions could add further value.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides 100% coverage for the single parameter, including the clarification that decision_id is a UUID and not a name. The description adds no further parameter semantics beyond what the schema already documents, so it meets the baseline for schema-heavy parameter coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Retrieve') and clearly identifies the resource ('generated DS2 execution code for a decision flow'). It distinguishes itself from sibling tools like get_decision_flow and get_decision_flow_revision by focusing specifically on the generated execution code, not metadata or revision details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used when the generated DS2 execution code for a decision flow is needed, but it does not explicitly state when to choose this over related tools such as get_decision_flow or get_decision_flow_revision. No exclusions or alternative usage guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that only locked revisions are listed, a key behavioral detail not captured by annotations. Annotations already mark it as read-only and idempotent, so the added context about locked revisions is valuable and non-contradictory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single clear sentence with zero filler. It is front-loaded and efficient, conveying all necessary information without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with an output schema present, this description is sufficient. It identifies the core purpose and relies on the schema for field details. It could optionally mention ordering or pagination nuances, but that is not critical given the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides comprehensive descriptions for both parameters (ruleset_id and limit), covering 100% of parameters. The description adds no extra parameter details, but the schema already supplies needed context, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and the resource (locked revisions of a rule set), distinguishing it from sibling tools like list_business_rulesets or get_business_ruleset. It is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. It does not mention that it is appropriate after lock_business_ruleset_revision or how it differs from listing all revisions. Usage context is implied but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the substring-filter behavior, but that is also present in the schema. It does not disclose other behavioral traits like pagination, ordering, or result scope beyond what the schema/annotations already provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that immediately states the action and resource. No filler or redundant text; every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with a strong output schema and comprehensive annotations, this description is sufficient. It identifies the resource and filter capability. Slight gap: it could clarify that results are a summary list (not full rule-set definitions), but the output schema likely handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% — both 'limit' and 'filter_name' are fully described with defaults and types. The description only repeats the substring filter concept without adding new meaning, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List'), a clear resource ('SAS Business Rules rule sets'), and a scope qualifier ('optionally filtered by name substring'). This distinguishes it from siblings like list_business_rules (different resource) and get_business_ruleset (singular retrieval).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies enumeration with an optional name filter, but it does not explicitly state when to use this tool versus alternatives such as list_business_rules or get_business_ruleset. There are no exclusions or direct comparisons, so guidance is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the description does not need to restate those. The phrase 'Download file content' adds slight behavioral context about the operation, but no additional detail about errors, format, or authorization is provided beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no filler or redundant information. It is perfectly sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only download tool with one well-described parameter, clear annotations, and an output schema, the description is fully sufficient. It tells the agent exactly what the tool does and leaves structured details to the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers 100% of parameter semantics with a clear description ('ID of the file to download'). The tool description adds no extra meaning beyond what the schema provides, which fits the baseline high-coverage case.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Download') and identifies the resource ('file content' from 'Viya Files Service'), clearly distinguishing it from sibling tools like list_files and upload_file. It unambiguously states what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this tool is used to retrieve file content, which is clear for a single-purpose download tool. However, it does not explicitly state when to use it over alternatives or mention any preconditions such as file existence or access permissions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is established. The description itself does not add behavioral context beyond the action of listing; it does not mention pagination, ordering, or any side effects. Given the annotations cover the core safety aspects, a score of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence with no redundant words. It is front-loaded with the action and resource, making it immediately clear and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with no parameters and comprehensive annotations, the description fully captures the tool's functionality. The presence of an output schema is mentioned in context but not detailed, which is not a deficiency for such a straightforward listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the input schema confirms an empty object. As per calibration, a baseline of 4 is warranted when there are no parameters, and the description does not need to explain any parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'List' with the resource 'available CAS servers' and clarifies the context 'on the Viya environment.' This clearly distinguishes it from sibling tools like list_compute_contexts or list_caslibs, which target different object types. It is not a tautology since it adds 'available' and the environment scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as list_compute_contexts or list_caslibs. It lacks any mention of scenarios, prerequisites, or exclusions, leaving the agent to infer usage solely from the resource name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so safety behavior is covered. The description adds the substring filter behavior, which is a useful extra. However, it does not disclose additional behavioral details such as ordering, pagination behavior, or access-scoping constraints, so it adds only modest value beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that directly states the action, resource, and optional modifier. Every word contributes value; there is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the 100% parameter schema coverage, the presence of an output schema, and effective annotations, the description is sufficient. It captures the core purpose and optional filtering without needing to explain return values or parameter details that are already structured.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters (limit and filter_name) already documented with clear descriptions. The description's mention of 'filtered by name substring' merely reaffirms the schema's filter_name semantics without adding new information or clarifying format, case sensitivity, or edge cases.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific resource ('SAS Intelligent Decisioning flows') and the action ('List'), plus an optional filtering capability. This distinguishes it from sibling tools like list_jobs and list_business_rulesets, making the purpose immediately unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (when you need to enumerate decision flows, optionally filtered by name), but it does not explicitly state alternatives or when not to use it. There is no mention of related tools like get_decision_flow or list_decision_flow_revisions for more targeted needs, so usage guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate a read-only, idempotent, non-destructive operation. The description adds the 'published' state scope, which is useful, but it does not describe pagination behavior, ordering, visibility limits, or what precisely is included in a module's data. It goes slightly beyond the annotations without being exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, compact sentence that front-loads the operation and resource. Every word contributes to understanding the tool's scope, and there is no repetitive or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a simple listing tool with one optional parameter, a complete input schema, an output schema, and annotations that fully describe its safe behavior. The description's explicit mention of 'published' adds the key business context needed for selection, making the overall package complete enough for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, limit, is fully documented in the input schema with type, default, and a clear description. Since schema description coverage is 100%, the description does not need to add much; it adds no redundant or conflicting parameter information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'List' with a clearly defined resource: 'published scoring models and decisions (MAS modules).' This distinguishes it from other list tools in the sibling set, such as list_jobs and list_reports, by naming both the content type and publishment state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies the use case—listing MAS modules that are published—but it does not explicitly explain when to choose this over related sibling tools like list_registered_models, list_ml_projects, or list_decision_flows. There are no exclusionary guidelines or alternative tool references.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already state readOnlyHint=true and destructiveHint=false, so the description doesn't need to cover safety. The description adds that it's a 'single rule's definition' but doesn't disclose what happens if not found or if it returns null. With annotations providing the read-only profile, this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, to the point, no filler. Structure is optimal for a single-purpose tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is an output schema, so description need not detail return values. For a simple fetch with two parameters and high schema coverage, the description is complete enough. Minor gap: doesn't mention how to get the rule_id (probably from list_business_rules), but that's not required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is high (100%), so schema already defines both parameters. The description adds no extra meaning beyond the schema, but it's consistent. Baselines 3 for high coverage is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb 'Fetch' + specific resource 'rule's definition from a SAS Business Rules rule set'. Distinguishes from siblings like list_business_rules (which lists many) and get_business_ruleset (which fetches the whole ruleset). The singular intent is explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clear context of fetching a single rule's definition, and the parameter 'ruleset_id' notes it's the parent rule set UUID. However, it doesn't explicitly when to use versus alternatives like list_business_rules or get_business_ruleset, though the name implies it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations thoroughly declare the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), matching the description's 'Fetch' semantics—no contradiction. However, the description adds no behavior details beyond what annotations already declare (e.g., not-found handling, auth requirements, or rate limits). Given the strong annotation coverage, the bar is lower, and the description neither undercuts nor enriches it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded 9-word sentence. Every word contributes: the verb, the resource, and the identifier scope. No fluff, no repetition of the tool name, and no wasted characters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-parameter read operation with a supplied output schema, the description is nearly sufficient. Sibling differentiation via the parameter annotation is helpful. The only gap is the lack of explicit not-found/error behavior, but given the tool's simplicity, full annotation coverage, and presence of an output schema, this is a minor miss.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema claims 100% parameter coverage, and the ruleset_id parameter description is genuinely informative: it identifies the expected format (UUID), explicitly states what it is not (a name), and cross-references list_business_rulesets as the source of the ID. Per the rubric, high schema coverage earns a 3 baseline; the useful cross-reference and format clarification push it to a 4. Only the lack of any additional meaning contributed by the tool description itself prevents a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Fetch a single SAS Business Rules rule set by ID' uses a specific verb (Fetch), a specific resource ('SAS Business Rules rule set'), and the identification method ('by ID'). This precisely distinguishes it from sibling list_fetch operations and clearly communicates the singular-fetch scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description itself does not provide explicit when-to-use guidance or exclusions relative to alternatives. However, the parameter description adds valuable disambiguation: it clarifies that a UUID is required (not a name) and points to list_business_rulesets as the source of both, which implicitly differentiates this get-by-ID pattern from the list tool. Explicit usage heuristics would have scored higher.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, and the description adds valuable behavioral context beyond that: missing tables return a structured not_found with two specific causes instead of a raw HTTP error. This informs the agent about error handling and edge cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two efficient sentences: the first states the core purpose and output, the second adds a targeted edge-case behavior. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only metadata tool with rich annotations, a complete input schema, and an output schema, the description provides purpose, output shape, and key error behavior. It lacks explicit comparison with sibling tools, but the name and content are sufficient for correct selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with clear descriptions, so the description does not need to add parameter-level detail. It neither repeats nor expands on the schema, but the schema is sufficient, supporting the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Get') and resource ('column metadata for a CAS table') and enumerates the exact output fields (names, types, labels, formats). This clearly distinguishes it from sibling tools like get_castable_data or get_castable_info.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the descriptive name and the stated purpose, but there is no explicit guidance on when to use this tool versus alternatives such as list_compute_columns or get_castable_info. The error-behavior note is useful but does not address tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, covering the safety profile. The description adds the types of metadata returned (row count, column count, size), which slightly extends beyond the annotations. However, it doesn't describe error behavior, permissions, or other edge cases; given annotations handle safety, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-front-loaded sentence with no filler. It efficiently states what the tool does and gives examples of returned metadata. Zero wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a simple signature (3 required parameters), good annotations, and an output schema that presumably details the metadata fields. The description sufficiently conveys the return concept (row count, column count, size) without over-explaining. It doesn't mention prerequisites or error cases, but for this low-complexity read operation, it's largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter is already documented in the input schema. The description does not add any extra semantics beyond what the schema provides (e.g., it doesn't hint at how parameters are used). Per rubric, baseline 3 applies when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get metadata') and a clear resource ('a CAS table'), listing concrete examples (row count, column count, size). It clearly distinguishes from siblings like list_castables (lists tables) and get_castable_columns (gets columns) by focusing on table-level metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage: it's for a single named table, not for listing. It doesn't explicitly say when not to use it or name alternatives, but the context is clear from the wording. No exclusions are stated, but that's acceptable for a straightforward read tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint: true, idempotentHint: true, and destructiveHint: false, covering the operation's safety profile. The description adds the behavioral constraint that the job must be completed, which is not in the annotations. It does not describe what happens if the job is not completed, but the additional context is valuable beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that clearly states the action and object. There is no redundant information, fluff, or unnecessary detail. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and an output schema, the description is nearly complete. It identifies the purpose, the prerequisite of a completed job, and the safety profile via annotations. Minor gaps (e.g., behavior when job is not completed, or content of the log) are largely covered by the output schema and the simple nature of the operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the only parameter, job_id, is described as 'ID of the job.' The tool description itself does not elaborate on this parameter, but since the schema fully documents it, the baseline of 3 applies. The description adds no extra semantic beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Retrieve' and identifies the resource as 'the log of a completed job'. This clearly distinguishes it from siblings like get_job_status (status vs. log) and list_jobs (list vs. single log). The phrase 'completed job' adds specificity beyond a generic 'job log'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for completed jobs only, providing a conditional usage context. However, it does not mention alternatives or explicitly state when not to use it (e.g., for running jobs use get_job_status). The guidance is only implicit, not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already communicate that this is a non-read-only, destructive, non-idempotent operation. The description adds useful context about asynchronous execution and the Job Execution service, but it does not disclose the potential side effects of executing arbitrary SAS code or any lifecycle implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with a front-loaded verb and no wasted words. It delivers the essential information immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 2-parameter tool with an output schema and annotations, the description is adequate: it states what the tool does and how it executes. It could optionally mention tracking via get_job_status or cancel_job, but those sibling tools make lifecycle management discoverable, so the description is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters (sas_code and job_name) are already documented in the schema. The description itself adds no parameter-level semantics, earning a baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Submit' with a clear resource ('a SAS job') and explicit mode ('asynchronous execution via the Job Execution service'). This distinguishes it from sibling tools like execute_sas_code, which likely runs synchronously.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies this tool is for asynchronous job submission, which is useful context for when to use it versus execute_sas_code. However, it does not explicitly state when not to use it or name alternatives, so it stops short of full guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true, so the safety profile is known. The description adds no additional behavioral context (e.g., that the update overwrites the entire rule or requires all fields to be provided). Given the annotations cover the essential traits, the description is adequate but does not enrich understanding beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no redundancy. It is front-loaded with the core action and resource, and every word contributes to the meaning, achieving maximum efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity (7 required parameters) and the presence of a detailed input schema plus an output schema, the description is sufficient. It specifies the resource type and operation, and the schema covers parameter semantics. It does not explain edge cases or prerequisites, but the schema and annotations provide adequate context for a simple update operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for all 7 parameters, so the schema alone clearly documents each field. The tool description does not provide any additional parameter-level meaning or context, matching the baseline of 3 for high coverage without extra description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Update') and clearly identifies the resource ('existing rule inside a SAS Business Rules rule set'), aligning with the tool's name and distinguishing it from sibling operations like create or delete. It is unambiguous and specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for updating an existing rule, and the tool name plus required rule_id and ruleset_id make the context clear. It does not explicitly mention alternatives or exclusions, but the purpose is evident from the wording and sibling tools, meeting the criteria for clear context without explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already assert readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description only states that it lists facets or values, which is read-only by definition. It adds no further behavioral context (e.g., limits, paging) beyond what annotations suffice. It doesn't contradict annotations, but also doesn't provide extra value. Given annotations cover the safety profile, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (about 4 sentences), with a clear opening sentence that sums up the purpose. It proceeds to explain the two usage modes and ends with the practical outcome. No superfluous text, every line has value. Excellent structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with only three parameters, all documented in schema, and an output schema provided, the description is fairly complete. It explains the primary two modes (list facets vs list values) and ties to catalog_search queries. It doesn't cover the 'limit' and 'query' details in the description, but they are fully described in the schema. The tool's simplicity and having output schema we have, the description is sufficient. Minor deduction for not mentioning how query interacts with values/facets more specifically, but overall 4.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes all three parameters with 100% coverage, so high baseline. The description adds context for the facet parameter (calling with no facet vs with a facet name) and clarifies that values are 'suggested/valid' values, which enriches the schema's dry description. It also explains the query parameter as an optional filter, matching the schema. This adds meaning beyond the schema, especially for facet's dual behavior, so 4 is justified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'list facets, or values for one facet' and explicitly ties it to catalog_search. It distinguishes from sibling tools like catalog_find_instance by referencing catalog_search queries and saying
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage instructions: calling with no facet lists facets; calling with a facet name gets values. It also tells the user to use the results to build precise catalog_search queries, indicating when it's appropriate (before a search). Does not explicitly mention when not to use or alternatives, but the context is clear enough. Slight deduction for not naming the alternative (like using catalog_search directly), but the intent is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations readOnlyHint=false and destructiveHint=true, the description adds context that copying creates a new report without modifying the original, and it describes the return payload. It does not elaborate on permissions or other side effects, but the annotations already provide safety hints, so this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, two sentences, with no redundant content. It effectively conveys the purpose, usage context, and return format without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for this simple copy tool: it states what it does, when to use it, and what it returns. The annotations and schema descriptions cover the remaining context, so no critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already has descriptions for all four parameters (name, folder, report_id, on_conflict), so the description does not add further parameter details. The description does mention the return structure but that is output-related, not parameter semantics. Thus the baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool copies a Visual Analytics report and returns the new copy's id. It is specific and distinguishes from sibling tools like create_report, delete_report, and export_report by focusing on the copy operation and its typical use cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit use cases (tailoring reports, copy-and-replace pattern) and references the companion tool apply_report_operations for the replace step. However, it does not explicitly mention when not to use this tool or alternative tools, so it falls just short of a perfect score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are generic (all false), so the description carries the burden of behavioral disclosure. It adds the critical fact that an empty rule set is unusable and requires subsequent population, which goes beyond annotation hints. While it doesn't describe side effects like permissions or response behavior, this is sufficient for a create operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise, consisting of two sentences that front-load the purpose and immediately follow with a crucial usage constraint. No redundant or wasteful text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema and simple parameters, the description provides necessary context including the dependency on create_business_rule. It is complete for its intended purpose, though it could mention error cases or prerequisites if any existed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents parameters. The description adds no additional parameter-specific semantics beyond the schema, which is acceptable per baseline expectations.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and specific resource 'SAS Business Rules rule set', distinguishing it from sibling tools like list, update, and delete. The added note about rules needing population further clarifies its distinct role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly explains that a rule set without rules cannot be used in a decision flow and advises following up with create_business_rule, providing a clear workflow hint. It does not explicitly state when NOT to use it or list alternatives, but the guidance is practical and contextual.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are all false (readOnlyHint=false, destructiveHint=false), indicating this is a mutation operation but not destructive. The description adds context about the creation process (chaining rule set steps) and references lock_business_ruleset_revision for versioning, which is useful. It doesn't detail side effects (e.g., what happens to existing revisions), but given annotations are all false, this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that front-loads the core purpose. It avoids redundancy and is appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having a detailed schema (100% coverage) and an output schema, the description correctly explains the core workflow (chaining steps) and references a related tool (lock_business_ruleset_revision). For a creation tool with a rich schema, this is sufficient. It could mention relationships to other tools like update or publish, but the schema covers most operational details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with each parameter having a detailed description. The description adds minimal value over the schema, as it only rephrases the 'rule_set_steps' concept. The schema already explains the structure of signature and rule_set_steps parameters, so the description doesn't add significant extra meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Create') and resource ('SAS Intelligent Decisioning flow'), and specifies the chaining of rule set steps. It distinguishes itself from sibling tools like create_business_ruleset and update_decision_flow by focusing on the flow creation with step chaining.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it (to create a decision flow) but does not explicitly state when not to use it or mention alternatives. However, the context of sibling tools (e.g., update_decision_flow for modifications, publish_decision_flow for publishing) provides clear usage context. Lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark destructiveHint=true, readOnlyHint=false, and idempotentHint=true. The description adds the important context that the deletion is permanent/irreversible and scopes the deletion to a rule rather than the parent rule set. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, front-loaded sentence states the verb, object, and scope with no filler or repetition. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive operation, the combination of annotations, complete parameter documentation, output schema, and the 'permanently' qualifier provides adequate context. It could mention prerequisites such as revision locking, but this is not explicitly required and would be speculative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are already well documented (rule_id as the rule UUID and ruleset_id as parent UUID, with name/id clarification). The prose description adds no additional parameter-level meaning, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses a specific verb 'delete' with an explicit resource: 'a rule from a SAS Business Rules rule set.' It clearly differentiates from sibling delete_business_ruleset and other delete tools by naming the exact object and scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clear context: use this to permanently delete a rule within a ruleset. It does not explicitly list alternatives or exclusions (e.g., delete_business_ruleset for the whole ruleset), but the resource scope is unambiguous and implies the correct situation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description need not repeat safety. It adds the behavioral detail about the reusable per-user compute session, which is useful for understanding execution environment. No contradictions; no additional disclosure needed for a simple read-only tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the primary purpose, the second provides execution context. No wasted words, front-loaded with the core action, and perfectly sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the rich schema (100% parameter coverage), strong annotations (read-only/idempotent), and presence of an output schema, the description is sufficient. It covers the essential operation and compute-session context. It could mention pagination behavior, but the schema's 'limit' and 'start' parameters already capture that, and output schema handles return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter (limit, start, table_name, filter_name, library_name, compute_context_name) clearly described. The tool description does not add semantic detail beyond the schema, which is the baseline 3 per the rubric.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'List the columns of a table in a SAS library within a compute context.' This identifies the specific resource (table columns), the scope (compute context), and distinguishes it from sibling tools like list_compute_tables or get_castable_columns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description adds context with 'Runs in the reusable per-user compute session for the context,' which helps the agent understand when this tool is appropriate (compute sessions) and implies it is not for CAS tables. It does not explicitly name alternatives or exclusions, but the context is clear enough for a read-only list operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds useful context about the reusable session and visibility of prior execute_sas_code calls, enhancing transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise, focused sentences with no redundant information. Well-structured and directly relevant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core purpose and session behavior, but lacks details about return format or possible errors; however, for a simple list operation, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides full descriptions for all four parameters (100% coverage), so the description adds no extra parameter meaning. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool lists SAS libraries (librefs) for a compute context, distinguishing it from sibling tools like list_compute_contexts or list_caslibs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for listing libraries within a compute session and notes it runs in the reusable per-user session, but does not explicitly contrast with alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint=true and idempotentHint=true annotations already present, the bar for description-only behavioral disclosure is lower. The description adds the key detail: it returns only source tables not yet in CAS memory and that they are the raw set for promote_table_to_memory. No contradiction with annotations (consistent with read-only, non-destructive). It could optionally mention error behavior for a bad caslib, but the behavioral core is covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A tight, two-sentence description with zero wasted words. It front-loads the core purpose in the first line, then immediately connects it to the downstream tool. Fully earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of a full output schema plus annotations, the description is complete for the agent to select and invoke it. A minor deduction for not explicitly describing the 'limit' semantics beyond the schema, but overall it's a well-rounded, sufficient description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% — all three parameters are well-documented in the schema (e.g., 'Maximum number of tables to return (default 50)'). The description accurately sets up the domain (CAS library, caslib) but adds no new parameter-specific detail beyond the schema. Baseline 3 is correct here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Purpose is explicitly stated with a specific verb+resource: 'List source tables that are NOT yet loaded into memory in a CAS library.' The critical qualifier 'NOT yet loaded into memory' separates it from sibling tools like list_castables or list_compute_tables. It clearly identifies the exact resource (source tables in a caslib) and the specific state (pre-memory).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly declares this is 'the candidates for promote_table_to_memory,' which is the key guidance for a user to know when this is the right tool. It could be a 5 but doesn't explicitly say 'use X for in-memory tables instead' or name an alternative by its full tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds important behavioral context beyond the annotations: it mentions the tool is idempotent (which aligns with the idempotentHint annotation), but also explains the specific scope (global visibility to all sessions) and that it uses the casManagement updateTableState API. It discloses that it loads from a caslib data source, which is a state-changing operation. The annotations already indicate readOnlyHint=false and destructiveHint=false, but the description provides additional context about the global scope and idempotency behavior, which is valuable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of three short sentences that front-load the main purpose, then explain the mechanism and idempotency, and finally provide a hint for discovery. No unnecessary words. Well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool complexity (no nested objects, 3 required params, has output schema, and annotations), the description is quite complete. It explains the scope, idempotency, and suggests a complementary tool for discovery. However, it doesn't mention any potential errors or if the operation requires special permissions, but given the output schema exists and annotations cover basic safety, this is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes all three parameters with reasonable descriptions (server_id, table_name, caslib_name). The description does not add further detail about the parameters beyond what the schema provides, but it does clarify that the table is loaded from a caslib and promoted to global scope. Since schema coverage is 100%, the schema does the heavy lifting, and the description adds little extra semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool loads a source table into CAS memory at global scope, using a specific API, and differentiates from siblings like list_source_tables and get_castable_info. It specifies the resource (source table) and the action (load/promote). The purpose is unambiguous and distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that it loads from a caslib data source, promotes to global scope, and mentions idempotency. It explicitly suggests using list_source_tables to discover unloaded tables, providing some usage context. However, it doesn't explicitly state when not to use this tool compared to other options like upload_data or query_data, but does recommend the discovery tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds transparency about server-side behavior (reads from disk, fetches URL, handles binary files) and configuration flags. It does not contradict the annotations (readOnlyHint=false, etc.). Since annotations already exist, this extra context is valuable but not exhaustive, earning a 4.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with bullet points, but it is verbose and repeats much of the schema content. The bullet lists are helpful, but the text could be more concise without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers all essential usage aspects: content sources, target folder, constraints, and configuration sensitivity. Since an output schema exists, return values are not explained, which is acceptable. The description is complete enough for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description repeats parameter descriptions, but it adds meaningful context: clarifies that file_path is server-side, url is fetched, content is inline, and parent_folder_uri is a target folder. It also mentions size limits and binary safety, enriching beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool uploads a file to the Viya Files Service, optionally into a Content folder, and specifies three input methods. This is specific and distinguishes from siblings like upload_data or upload_inline_data by focusing on file-based uploads.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use each input method (content for inline text, file_path for server disk, url for remote) and notes constraints like size limits and the ALLOW_LOCAL_FILE_UPLOAD flag. However, it does not explicitly contrast with sibling upload tools, so guidance on tool selection is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the exact behavior: it returns a CSV of column metadata and profile, or if not profiled, a recommendation for adhoc analysis. This aligns with the annotations (`readOnlyHint: true`, `destructiveHint: false`, `idempotentHint: true`) and adds detail about the conditional output, which is transparent about side effects (none).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is succinct, with no redundant information. It is well-structured in three sentences, covering purpose, fallback behavior, and parameter usage. It avoids verbosity while providing necessary context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema (not shown), the description adequately covers the main functionality and edge case (unprofiled table). It explains the two possible outputs and the parameter resolution logic, which is sufficient for most use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description clarifies the relationship between `instance_id` and `resource_uri`, stating they are alternatives and that `instance_id` takes precedence. This adds meaning beyond the schema descriptions, which already cover the basic purpose of each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Download a catalog table's data dictionary and profile as CSV.' It explicitly distinguishes it from related tools by describing the fallback behavior when a profile is not yet available, recommending `catalog_run_adhoc_analysis`.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains when to use this tool versus an alternative: when the table is not profiled, it returns a recommendation to run `catalog_run_adhoc_analysis`. It also clarifies how to identify the table (via `instance_id` or `resource_uri`), giving precedence rules. However, it does not explicitly state general scenarios for choosing this over other catalog-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds valuable behavioral context beyond annotations by disclosing what each record reports (run status + metadata counts for tables enumerated/added/updated/removed), which tells the agent it's a pure history viewer. It also explains the functional relationship to catalog_run_agent. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tightly written sentences with zero filler. The first sentence states the core purpose; the second explains what the output contains and how to apply it. Every sentence earns its place, and the most important information (what the tool does) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With a comprehensive output schema present, the description doesn't need to detail return values. Annotations fully cover the safety profile (read-only, idempotent, non-destructive). The description adds the functional context — what the history reveals (status and metadata changes) and how to use it (confirming catalog_run_agent runs). For a simple history-retrieval tool, this is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with all three parameters (limit, start, agent_id) documented in the schema itself. The description adds minimal parameter detail beyond schema — it implicitly references agent_id via the catalog_run_agent relationship but doesn't explain limit/start semantics. The schema already handles parameter documentation, so the baseline-3 score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb+resource combination: 'Get the execution history of a catalog agent's runs' — clearly stating what the tool does. It distinguishes itself from sibling tools like catalog_run_agent (which executes runs) and catalog_list_agents (which lists agents) by focusing on history retrieval. The first sentence alone fully captures the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly ties usage to a concrete use case: 'so you can confirm a run started by catalog_run_agent finished and what it changed.' This names the initiating sibling tool and explains when the output is relevant. However, it doesn't explicitly state when NOT to use it or mention alternatives (e.g., get_job_status for async job tracking), leaving some room for improvement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description clearly discloses the asynchronous behavior: the job runs asynchronously, may take time, and requires polling with the returned id. It also describes what the job produces (data dictionary, column statistics, quality metrics) and the effect of the NLP parameters on asset metadata. With no annotations provided, this is solid behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose in the first sentenceholaWhat is the function of the first paragraph? It explains the job result and async behavior, and second paragraph explains NLP params. Two concise paragraphs, each earning their place—no fluff. Slightly verbose in the middle with the bracket list, but acceptable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the job lifecycle (submit, poll), references related tools, and clarifies the side effects of NLP parameters. The schema already documents parameters fully, and the output is a job id which is implicitly referenced. Complexity is moderate (async, multiple boolean params), and the description covers it well, though it could mention expected job duration or error handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is around 88% (7 of 8 params described in schema; resource_uri has a detailed example, the three booleans have detailed descriptions, name/description/provider are straightforward). The description adds crucial semantics for the three NLP booleans: they are enabled by default and drive informationPrivacy, nlpTags, etc., and explains when to disable them. This adds meaning beyond what the schema's terse descriptions ('default True') provide.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: submitting an ad-hoc profiling job for a catalog table, with a specific verb ('Submit') and resource ('table in the catalog'). It distinguishes itself from sibling `catalog_download_table_profile` by noting the job computes what that tool returns, and from `catalog_get_adhoc_analysis` by mentioning the job id polling for completion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool: to submit an ad-hoc profiling job, and how it fits with siblings (`catalog_download_table_profile` returns what this job computes; `catalog_get_adhoc_analysis` is polled with the job id). It also gives guidance on when to disable NLP parameters (plain profile only). It doesn't explicitly say when NOT to use it, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint: true and idempotentHint: true, but the description adds value by explaining that deletion fails if the rule set is still referenced, which is not in the annotations. This gives the agent crucial behavioral context. No contradiction with annotations; in fact, it reinforces destructiveHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the first sentence states the action, the second provides a critical usage condition. No wasted words. Could arguably be more concise, but it's efficient and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter), the description is sufficiently complete. It covers purpose, usage condition, and parameter semantics. Output schema exists, so return details aren't needed. For a delete operation with clear annotations, this is adequate and near-complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter 'ruleset_id', with the description noting it's a UUID and not a name, and referencing list_business_rulesets for retrieval. This adds value beyond the schema's type information by clarifying the identifier format and how to obtain it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Permanently delete a SAS Business Rules rule set.' It specifies the resource (rule set) and the scope (permanently). It distinguishes from sibling tools like delete_business_rule and delete_decision_flow by focusing on rule set deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Only call this once the rule set is confirmed unused by any decision flow.' It implies when to use (after confirmation) and warns about failure if still referenced. It doesn't explicitly mention alternatives, but the sibling context includes list_business_rulesets to confirm usage, which is implied. No explicit exclusions, but the condition is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds context by explaining what agents are and their relationship to runs, but could also mention filtering/pagination details. However, given the annotation coverage, it's still solid.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is brief, front-loaded with the main action, and adds only essential detail about what agents are and related tools. No wasted sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity is moderate (3 optional params, output schema present), the description is sufficient. It explains the tool's purpose, guides usage with related tools, but could slightly enhance with mention of typical use case (e.g., listing during planning). However, with output schema available and annotations covering safety, it's quite complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter well-described (limit, start, filter_name). The description adds the conceptual context that agents crawl sources, but doesn't add further parameter semantics beyond what schema already provides. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists SAS Information Catalog discovery agents and explains what agents do (crawl data sources, collect metadata). It goes beyond the name to describe the resource and its role, and it distinguishes itself from related sibling tools by referencing the run and history tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage context: agents are used to crawl sources, and it direct users to `catalog_run_agent` to start one and `catalog_get_agent_history` to see results. It effectively tells the agent when to use this tool for listing agents, not for running or examining history.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate this is a write operation (readOnlyHint: false) and non-idempotent, but the description adds crucial behavioral details: the API's lenience with incomplete expressions and the DS2 code generation issue, plus boolean comparison requirements. These go beyond the schema annotations to warn the agent about subtle API behavior, but could be even more explicit about side effects (e.g., does it overwrite or append?).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, front-loaded with the purpose, and uses detailed but necessary examples for expression syntax. Every sentence adds value: first sentence defines the action, second explains the expression nuance, third addresses a specific edge case (boolean variables). No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there's an output schema (not shown but flagged), the description focuses on inputs and behavior. With 6 required parameters and complex expression requirements, the description covers the critical usage details (expression syntax, boolean comparisons). It lacks a note on error cases (e.g., what happens if ruleset doesn't exist) but the annotations and schema cover the basics. For a write tool without idempotency, this is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds significant value by explaining the 'conditional' parameter semantics ('if' starts a new chain, 'elseif' continues) and the ruleset_id disambiguation (not by name). The syntax notes about expressions directly complement the conditions/actions fields. This is above the baseline because it clarifies meaning that the schema alone doesn't fully convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: create a new rule inside an existing SAS Business Rules rule set, with high specificity about the rule set context. It distinguishes itself from sibling tools like create_business_ruleset (which creates the rule set, not a rule) and update_business_rule/delete_business_rule by focusing on creation within an existing set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool (when you need to add a rule to an existing rule set) and even includes important usage notes about variable name syntax and boolean comparisons. However, it doesn't explicitly mention when NOT to use it or point to alternatives for related operations (like updating or deleting rules).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the operation as destructive and idempotent; the description adds the crucial behavioral detail that the entire flow is replaced, not partially patched. This goes beyond what readOnlyHint/destructiveHint alone communicate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences: the first states purpose, the second delivers the critical usage warning. No filler or redundant restatement of the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the annotations, 100% schema coverage, and presence of an output schema, the description is largely sufficient. It explains the one non-obvious destructive behavior. It could optionally reference lock_decision_flow_revision or how to fetch existing steps, but this is not necessary for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds meaningful parameter-level guidance for rule_set_steps by requiring all steps (existing + new) and explaining the full-replacement semantics, which is not evident from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Update') with a specific resource ('existing SAS Intelligent Decisioning flow'), and the word 'existing' clearly distinguishes it from creation tools. The full-replacement warning further sharpens the tool's scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly conveys that this tool updates an existing flow and must be given all rule set steps because the update is a full replacement. It does not explicitly name alternatives like create_decision_flow, but the 'existing' phrasing and full-replacement caveat provide clear contextual guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, which the description aligns with by describing a synchronous, read-only export operation. It adds useful details about format-specific constraints (e.g., image_size required for png/svg, exactly one object for data formats) and that it uses synchronous endpoints. These go beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear opening sentence, a bulleted list of formats with associated requirements, and 'Note' lines for parameter constraints. Every sentence provides necessary information without redundancy. It is concise for the complexity it covers, and front-loads the core purpose before detailing formats.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should clarify what the tool returns. It mentions 'export endpoints' and formats like 'package' as a .zip, but doesn't explicitly state the response type (e.g., binary content vs. download URL). It covers most operational details, making it highly usable, but this one omission prevents a perfect score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers all five parameters with descriptions, but the tool description significantly enriches meaning by explaining how each format affects parameter usage (e.g., options for PDF overrides, report_objects constraints per format, image_size format). This adds value beyond the schema's individual field descriptions, clarifying the combined semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Export a Visual Analytics report (or specific report objects) in any format the VA service exposes'. It specifies the verb (export), the resource (report), and the scope (whole report or specific objects), and lists all supported formats. It distinguishes itself from siblings like copy_report or get_report by focusing on export in various formats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides detailed context on when to use each format (e.g., 'csv' requires exactly one object, 'png' requires image_size), and implies export for downloading data rather than on-screen viewing. However, it doesn't explicitly mention alternatives like get_report or download_file for comparison, but the format-specific requirements serve as clear use-case guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable context about the return content (variable signature details) beyond the annotations. It does not contradict the read-only, idempotent, and non-destructive annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that deliver the essential information without redundancy. The structure is clear and front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description sufficiently explains what the tool does and why it is useful, given there is no output schema to describe. It covers the key aspects needed to understand when and how to use it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema descriptions for step_id and module_id are already informative. The tool description does not add extra meaning to the parameters beyond the schema's own descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action ('Fetch') and the resource ('MAS module step's input/output variable signature'). The mention of being called before score_data distinguishes its specific use case from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to call before score_data and explains the rationale (to know variable names, types, order, and expected outputs). This provides clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds valuable context by noting it 'Runs in the reusable per-user compute session for the context,' which explains the execution environment beyond what annotations convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose, and includes only essential distinctions and context. Every sentence earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema and annotations present, the description sufficiently covers purpose, scope, and execution environment. It does not need to explain return values or parameters because the schema and output schema already capture those details. This is a complete description for a listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers 100% of parameters with descriptions including examples like 'WORK' and 'SASHELP'. The description adds no extra parameter semantics beyond referencing WORK as a libref, which is already present in the schema. Baseline of 3 is appropriate given high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'List the tables in a SAS library within a compute context.' It specifies the resource type (SAS/Compute tables) and explicitly distinguishes from in-memory CAS tables by referencing list_castables, which differentiates it from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides an explicit alternative ('see list_castables') and clarifies the distinction between SAS/Compute and CAS tables. This direct when-not guidance helps the agent choose the correct tool. The mention of the reusable per-user compute session further contextualizes usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only indicate this is a non-read-only, non-destructive, non-idempotent operation. The description goes much further by revealing that the request replaces the rule set's full content, requiring a fetch-and-resend pattern, and explicitly warns that omitting rules would wipe the live rule set. This is critical behavioral disclosure beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, starting with the core purpose, then explaining context, and finally providing a critical implementation warning. Every sentence adds value, and the three short paragraphs are easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a complex and potentially risky behavior (replacing full content), and the description fully covers how to use it correctly, why revisions are needed, and what pitfalls to avoid. An output schema exists, so return value details are not needed. The description is complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The schema already explains both parameters (ruleset_id is a UUID, revision_type is minor/major). The description adds no extra parameter-specific meaning beyond what the schema provides, though the warning about the body content relates to how parameters should be used.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Lock') and resource ('current state of a rule set as an immutable revision'), clearly distinguishing this tool from siblings like lock_decision_flow_revision. It also explains the purpose of revisions in decision flows, making the tool's role unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states when to use the tool ('before wiring a rule set into a decision flow') and when to call again ('after editing rules'). It also warns about the risk of omitting rules. However, it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description conveys that the tool makes the state immutable and creates a snapshot, which is a key behavioral trait not fully captured by the annotations. It doesn't mention reversibility or idempotency, but the term 'immutable' implies a non-reversible freeze.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no redundancy, and directly conveys the purpose and usage. It is efficiently structured with a clear action statement followed by a contextual instruction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter, the description is complete: it explains what it does, when to use it, and how it relates to the publishing step. The existence of an output schema (not shown) suggests return details are handled elsewhere, so no gap is apparent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter decision_id is already well-documented in the schema (UUID, not name, with reference to list_decision_flows). The tool description adds no further semantic detail beyond what the schema provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action (lock), the object (decision flow), and the result (immutable revision). It is distinct from sibling tools like create, update, or publish by focusing on freezing a state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says to call after a successful create/update, and explains the purpose of creating a point-in-time snapshot for later publishing. This gives a clear when-to-use context without ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (destructiveHint, idempotentHint), the description adds valuable behavioral context: the server keeps a reusable session per user/context, state persists across execute_sas_code and list_compute_* calls, and the next call transparently creates a fresh session. This explains the consequences of resetting without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four sentences, front-loaded with the action, then providing necessary context about the session reuse model and the effect of calling reset. Every sentence contributes meaning without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a single optional parameter, well-documented in the schema, and an output schema exists (per context signals). The description fully explains the tool's purpose, the persistence behavior, and the consequence of resetting, making it complete for an agent to decide when and how to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already fully describes the only parameter (compute_context_name) with 100% coverage, including its default behavior. The tool description does not add any parameter-specific details, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: "Reset (delete) the cached compute session for a compute context." It clearly distinguishes this tool from its siblings by explaining it discards persisted SAS session state, which is unique among the listed compute-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool: "Call this to discard that state" after detailing the persistence mechanism. It provides clear context about the session caching behavior and the effect on subsequent calls, but it does not explicitly name alternative tools or state when not to use it. However, the usage context is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite annotations already indicating destructiveHint and readOnlyHint false, the description adds specific behavioral details: changing the signature can invalidate existing rules that reference removed variables. It also recommends a pre-check via get_business_ruleset, which clarifies the potential side effects beyond generic destructiveness. This is valuable extra context that the annotations do not convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise, with a single purpose statement followed by a focused caution. Every sentence earns its place, and the warning is front-loaded after the purpose. There is no redundant text or unnecessary detail, making it easy to scan and comprehend.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's mutation potential (destructiveHint true), the complex signature behavior, and the presence of a full output schema (not shown but indicated), the description covers all necessary aspects: it states the purpose, warns about risks, and advises a safeguard. The schema handles parameter details, so the description only needs to highlight the critical side-effect, which it does effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage for all four parameters, with detailed descriptions for each. The description does not add extra semantic meaning beyond what the schema provides; it only mentions the signature's role in invalidation, which is more behavioral than parameter-specific. Thus, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing SAS Business Rules rule set's name/description/signature, using a specific verb and resource. It distinguishes from siblings like create_business_ruleset, list_business_rulesets, and delete_business_ruleset by clearly implying modification of an existing entity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a contextual warning about signature changes invalidating rules and suggests checking the current ruleset with get_business_ruleset first. This offers guidance on safe usage, though it does not explicitly compare with alternatives or state when not to use it. The caution implicitly advises against blind updates, but the main usage context (updating an existing ruleset) is obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
All annotation hints are false, so the description carries the full informational burden. It adds valuable behavioral context: the payload-cost implication ('whole payload travels through the model's context'), the text-only constraint with explicit format list, and the architectural distinction that upload_data 'reads the bytes server-side.' It doesn't cover duplicate-name behavior on table creation, but the context it does add goes well beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, perfectly front-loaded with purpose, each subsequent sentence earning its place. The second and third sentences work together to define the boundary with upload_data without redundancy. Zero filler words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter creation tool with an output schema, the description is remarkably complete: it covers use-case boundaries, performance/size implications, supported formats, and fallback paths. The presence of an output schema covers return values, so the description need not explain them. No significant gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description reinforces the data parameter's size sensitivity (hinting why a small inline string is appropriate) and echoes the data_format values (csv/tsv). However, it does not add meaningfully new parameter-level insight the schema descriptions don't already convey, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource+mechanism: 'Create a small CAS table from inline delimited text passed as a string.' It clearly identifies the resource (CAS table), the input mechanism (inline delimited text), and scopes it as 'small,' which distinguishes it from sibling upload_data. The qualifier 'small' also pre-empts misuse.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Excellent explicit guidance: 'Use this only for tiny, hand-built tables — a lookup/mapping table the model constructs on the fly, or a quick test table — because the whole payload travels through the model's context as a tool argument.' It gives concrete when-to-use examples (lookup tables, test tables), the underlying reason (context size), and explicitly names the alternative tool and why: 'For anything larger, or any file you already have, use upload_data (file_path/url), which reads the bytes server-side instead.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a read-only, idempotent, non-destructive operation. The description adds crucial behavioral nuance about the timing lag—that terminal status does not guarantee data availability—and explains the added fields (profile_ready and information_privacy), which is valuable context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is slightly longer than minimal but front-loads the purpose and then explains the polling nuance and readiness gates. Each sentence earns its place, though it could be tightened slightly without losing essential detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, the description does not need to detail return values. It thoroughly covers the behavioral context, the exact feedback loop (polling until ready), and cross-references the related download tool, making it complete for a polling/status tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter 'job_id' is fully described in the input schema (100% coverage), and its origin from 'catalog_run_adhoc_analysis' is also mentioned in the schema. The description does not add additional parameter-specific meaning, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves the status of an ad-hoc analysis job and additionally reports profile readiness. It distinguishes itself from sibling tools like 'get_job_status' by focusing on the ad-hoc analysis context and the readiness flag, making it unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is given: the job reaching terminal status is insufficient, and users should poll until 'profile_ready' is true before downloading. It also references the same gate used by 'catalog_download_table_profile', providing a clear when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond the destructiveHint annotation by explaining the persistent compute session behavior, leftover state risks, and the CAS libname preference. This is critical behavioral context that an agent would otherwise not know, and it does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear purpose sentence, process explanation, a highlighted IMPORTANT warning, and a useful tip. It is a bit lengthy but every sentence contributes meaningful information, so it earns a high score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and destructiveHint annotation, the description covers the tool's purpose, process, side effects (state persistence), and includes practical usage advice. It is fully complete for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% coverage for both parameters. The description adds value by explaining the practical implications of fresh_session in the context of state persistence, effectively enriching the parameter semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool executes SAS code in the Viya environment and returns job information, outlining the process (create job definition, execute, retrieve results). This specifically distinguishes it from siblings like submit_batch_job, which likely handles batch submission differently.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to pass fresh_session=True versus using reset_compute_session, and gives a practical CAS tip. It does not explicitly contrast with submit_batch_job or other execution alternatives, but the state-persistence warning effectively guides when a clean slate is needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the baseline is low. The description adds valuable behavioral context: it reduces the stored report to a compact outline, mentions possible return statuses (ok, not_found, outline_failed), and explains the output structure. This goes beyond the annotations and helps the agent understand the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is several sentences long but well-structured with a clear primary purpose, usage examples, and return format. Each bullet adds specific value (mapping to other tools). It is slightly longer than necessary but avoids redundancy and is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, read-only, idempotent), the description is complete: it explains the purpose, what the output contains, how to use it with other tools, and error conditions. The output schema is not shown but the description explicitly mentions the return structure and statuses, so no critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single required parameter report_id, and the schema description states 'The report id to outline.' The description does not add further detail about the parameter beyond what the schema already provides. Since the parameter is simple and unambiguous, the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool reads a report's structure (pages → objects) and provides the handles needed by other tools. It clearly distinguishes itself from siblings like get_report (full report) and describe_report_objects, and explains the compact outline nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit use cases: to edit an existing report, recover object names after an apply, or check batch results. It also maps the returned fields to specific parameters of other tools (e.g., object name → relativeToObject, label → export_report's report_objects, page label → page placement), giving clear guidance on when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive behavior, so the description does not contradict them. It adds value by detailing the returned information (analysisTimeStamp, informationPrivacy, nlpTerms, etc.), which goes beyond the safety annotations and clarifies the tool's side-effect-free nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and free of unnecessary fluff. It covers purpose, usage context, and output details in a clear, logical flow, using backticks for code identifiers. Every sentence contributes value, and the length is appropriate for the information conveyed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Since there is no output schema provided, the description compensates by explicitly listing the returned fields (id, analysisTimeStamp, informationPrivacy, nlpTerms, nlpTags, mostImportantFields). It also situates the tool within a workflow (before catalog_download_table_profile), making the overall context complete enough for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes the resource_uri parameter, achieving 100% coverage. The description augments this by explaining the origin of the URI (from a search hit or CAS table path) and its role in the lookup, which adds practical context beyond the schema's generic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Resolve the catalog instance for a source-asset URI.' It differentiates from sibling tools like catalog_search by specifying that it operates on an existing resource URI, and it lists the specific output (id and profile attributes).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: when you already have a resource URI (from a search hit or CAS table path) and need to obtain the instance id and profile attributes before calling catalog_download_table_profile. It also contrasts with catalog_search, which is for finding assets by free text/facets, providing clear usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, so safety profile is clear. Description adds value by explaining that it returns resource_uri for acting on live assets, that attributes map includes typical metadata fields, and that query uses a specific grammar with wildcards and facets. It doesn't mention pagination behavior beyond limit/start params, but those are in schema, so fine.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with clear sections: purpose in first line, then details on hits, then query grammar. No fluff; every sentence adds useful information. Grammar examples are compact and helpful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (search grammar, facets, connections to other tools), the description is thorough. It covers the search syntax, uses examples, mentions the helper tool, and explains the return structure (resource_uri and attributes). Output schema exists so return values don't need expansion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, all 4 params have descriptions. Description adds significant meaning by explaining the query grammar in detail (wildcards, facets, ranges, combinations), and mentions that `limit` should be used to control hits. It also notes `*` matches all names, which isn't in schema. Though params are fully described in schema, the description enriches understanding of the `query` param especially.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it searches the SAS Information Catalog for assets, enumerates asset types, and distinguishes its scope from tools like get_report or get_castable_data by explaining it finds assets without knowing server/library. It distinguishes from sibling tools like catalog_find_instance and catalog_search_helper, and explicitly mentions the resource_uri that connects to other tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance: use when you don't know the server/library first. It also references a sibling tool `catalog_search_helper` to discover valid facets, giving an alternative for discovering query terms. Clear exclusions implied since other tools handle specific asset operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate it's not read-only (readOnlyHint=false), so the description need not repeat that. It adds context about the verification step ('This tool verifies that first and returns an actionable error otherwise') and explains the URI construction from server_id/caslib_name/table_name. This goes beyond annotations and helps the agent understand side effects and dependencies.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tightly worded, with the core purpose stated first, followed by essential prerequisites and guidance. There is no fluff; every sentence contributes value. The structure is clear and front-loaded, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (9 params) and the existence of an output schema, the description covers the critical aspects: prerequisites (table in global scope), verification behavior, and URI construction. It doesn't need to explain return values since an output schema exists. The description is complete for the agent to understand when to use it and what it does.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all parameters have descriptions. The description adds extra meaning by explaining that the training table must be loaded globally and that the URI is built from server_id/caslib_name/table_name, which clarifies the relationship between those parameters. This added context goes beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Create a new AutoML pipeline automation project from a CAS table.' It specifies the resource (CAS table) and the purpose (AutoML pipeline automation), and it distinguishes itself from sibling tools like run_ml_project (running a project) and list_ml_projects (listing). This is a specific verb+resource pair with clear differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit prerequisites: 'The training table must already be loaded into CAS memory at **global** scope.' It even names alternative tools for preparation: 'use promote_table_to_memory to load + promote a source table, and list_source_tables to find one.' This gives clear when-to-use and when-not-to-use guidance, and points to alternatives for those prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the destructiveHint annotation, the description discloses non-obvious behaviors: the atomic creation with operations, the risk of leaving an empty report behind if later edits fail, VA prepending an empty default Page 1, and the result's verify_hint. These details meaningfully exceed what the annotations alone communicate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the main purpose and each sentence earns its place by adding actionable detail: return payload, atomicity rationale, follow-up editing, and export verification caveat. It is dense but not bloated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutating report-creation tool with four parameters, the description covers the full workflow: create, identify what was created, edit later, and verify output. It also includes a non-obvious export caveat, while structured annotations and output schema cover the remaining context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers all four parameters at 100% coverage, which provides the baseline. The description adds value by explaining that operations uses the same shape as apply_report_operations, is applied atomically at creation, and that describe_report_objects should be consulted for formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Create a Visual Analytics report') and states that the returned id is for further edits. It also distinguishes the empty-shell creation mode from the atomic full-build mode, which separates this tool from siblings like apply_report_operations and copy_report.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear workflow guidance: pass operations to build the whole report atomically at creation, then feed the returned id to apply_report_operations for further edits. It also tells the agent to verify page-by-page with export_report due to the prepended default Page 1. It does not explicitly enumerate alternatives like copy_report, but the context is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Although annotations already indicate destructive and idempotent behavior, the description adds crucial context: 'There is no per-object undo in the report API.' This explains irreversibility beyond just a destructive hint and also discloses the response format, which is not fully captured by the annotations. No contradictions with annotations are present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that state the core action, the key behavioral caveat, and the return values. It is front-loaded and every word adds value, with no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive operation with one parameter, the description covers all essential aspects: what is deleted, the irreversibility, the copy alternative, and the response payload. The output schema exists and handles return details, so no additional completeness gaps are evident.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage for the single required parameter report_id, with a clear description. The tool description adds no additional meaning or format details beyond what the schema already provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Delete a Visual Analytics report and its content', using a specific verb and resource, clearly distinguishing it from sibling tools like copy_report or create_report. It also clarifies the scope (report plus its content), leaving no ambiguity about what action is performed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'deleting and rebuilding (or copying first) is how you discard an unwanted report.' It also implies the alternative of copying first as a safeguard, and warns about the lack of undo, providing clear guidance for a destructive operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds valuable context beyond those hints: it 'reads a bundled catalog (no network),' so it is a low-cost operation with no external side effects. This explains why the tool is safe and cheap to call repeatedly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose and followed immediately by usage guidance and a cost note. Every sentence earns its place; there is no filler or repetition of schema or annotation details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity, full schema coverage, and strong annotations, the description is complete: it states what the tool does, when to use it, what it reads, why it is cheap, and how it relates to apply_report_operations. The presence of an output schema also removes the need to describe return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and each parameter already has a meaningful description in the schema, including examples and purpose. The tool description does not add parameter-specific semantics, but it does provide helpful framing by mentioning 'object's data roles' and catalog categories. Baseline 3 is appropriate because the schema carries the parameter documentation burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific statement: 'Discover what a Visual Analytics report can contain — operations and objects.' It explicitly names the consumer of this knowledge ('before calling apply_report_operations'), which distinguishes this tool from its siblings by stating its role as a preparatory discovery step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage guidance: 'Call this to learn how to build a report before calling apply_report_operations.' It also positions the tool as the cheap, offline alternative to guessing at data roles, making it clear when an agent should prefer this lookup over trial and error.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (all false), it discloses asynchronous execution, background result application, and the limitation of not stopping a run, which are critical behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant information, effectively conveying purpose and constraints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple schema and output schema existence, the description provides sufficient context: what it does, how it behaves, and what it does not do. No major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter agent_id is fully described in the schema, and the description adds context by referencing catalog_list_agents for identification. Coverage is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool starts a catalog discovery agent run, with specific action (crawl data source, populate/refresh metadata) and distinguishes from sibling tools like cancel or get history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly notes asynchronous behavior and that it cannot stop a run, implying when to use. However, it does not explicitly mention alternatives or provide a direct 'use this when' statement, though the sibling context clarifies.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behavioral traits beyond annotations: asynchronous publishing, polling the publish job until terminal state, server-generated module ID (not publish_name), and that rule set code generation errors are environment-level. These details add significant context that annotations alone do not convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured into four focused segments: purpose, prerequisite, async behavior, and error guidance. Every sentence provides useful information without fluff, and the key purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description covers the operational lifecycle (prerequisite, async polling, return integration with downstream tools), error interpretation, and the key conceptual pitfall about module ID vs publish_name. It is complete for a complex publish operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema covers all 5 parameters with descriptions, so baseline is 3. The description adds meaningful clarification about publish_name not being the MAS module ID and the returned moduleId being the real one, plus the polling behavior via jobUri. This extra semantic context elevates the score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource+destination: 'Publish a locked decision revision to a Micro Analytic Score (MAS) destination.' This clearly identifies the action and distinguishes it from sibling tools like list_mas_modules (listing) and score_data (scoring).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool: 'Required before score_data can execute the decision — MAS runs published modules, not decision flows directly.' It also tells the agent it can avoid a separate lookup via list_mas_modules, and provides troubleshooting for environment-level errors, giving clear context vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses behavior beyond the annotations (destructiveHint=true) by detailing atomic application, validation before HTTP calls, ETag optimistic-concurrency with retry, write-once placement/dataRoles, auto-naming and auto-sizing, and the save-as behavior to avoid destructive edits. It also explicitly states 'the whole batch is applied atomically (all succeed or nothing changes).' This is far more than annotations provide and is transparent about mutations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Despite its length, the description is well-structured with clear sections: an opening shorthand of usage, operation keys, layout & titles specifics, validation and concurrency behavior. Every sentence carries purpose—no fluff. It is front-loaded with the core verb and resource, and the technical details are organized logically for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of this tool (multiple operation types, layout rules, validation logic), the description is exceptionally complete. It covers operational invocation, validation guarantees, concurrency handling, and the output options (dry_run, result_report_name for save-as, response_format). It also references output verification via export_report and provides a comprehensive example. The presence of the output schema further reduces the need for result-format discussion, but the description still addresses all needed context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100% and the schema is rich, but the description adds even more semantic depth: it explains the operations array structure with an example, clarifies layout and placement options with concrete JSON snippets, and describes constraints like 'relativeToObject' against existing objects and the write-once rule. This exceeds the schema's role descriptions and gives crucial contextual meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource (apply operations to a report) and accurately characterizes the tool as the 'authoring workhorse' for adding pages, objects, parameters, and swapping data sources. It distinguishes itself from sibling tools like create_report, describe_report_objects, and get_report_outline by clearly scoping its role in the batch modification workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'This is how you add pages, add objects... set parameters, and swap data sources,' and gives detailed operational patterns. It provides when-to-use and when-not-to-use (e.g., page/report headers accept ONLY control objects, same-batch forward references fail). It names complementary tools (describe_report_objects, get_castable_columns, get_report_outline, export_report) and explains their roles for contextual layering.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Although annotations are all false, the description fully discloses behavior beyond structured fields: it states 'nothing is persisted—the result is materialised into session scratch, read back, and dropped,' explicitly refuses DDL/DML, and explains that a malformed LIMIT is silently discarded by CAS and that create_view_sql is never executed here. This is comprehensive transparency that exceeds what annotations alone provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every sentence earns its place, using bold headings for tier selection and dialect notes. It front-loads the core purpose, then progressively adds necessary nuance without fluff. Markdown formatting aids scanning, and the structure mirrors the user's decision flow (which tier, what dialect, how paging works).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool spanning two storage tiers with a non-standard dialect, the description covers all critical aspects: tier selection, identifier qualification, dialect unsupported features (CTE, MERGE), concatenated libref invisibility, row capping, and stable paging. Given the existing output schema, this is exceptionally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds substantial meaning: target is explained with namespace distinctions and examples, limit behavior is clarified (the tool enforces it, SQL LIMIT is ignored), and create_view_name is detailed as returning SQL text rather than executing. This goes far beyond the schema's short descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise verb-resource statement: 'Run a FedSQL SELECT against CAS or compute data and return the rows.' It immediately clarifies it handles both storage tiers with a single SQL surface, which distinguishes it from sibling tools like list_castables or execute_sas_code. The scope is unambiguous and unique.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit tier-selection guidance via 'target' with concrete qualification examples (Public.HMEQ, WORK.SALES). It details when NOT to use it (concatenated librefs like SASHELP/MAPS) and instructs to copy to WORK first, even suggesting alternative tool execute_sas_code for staging. Also covers dialect limitations (no CTE, no MERGE) and row-capping behavior, making usage conditions crystal clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that data is read server-side and never passes through the model context, and it explains the file size limitation. It also details how the URL parameter is handled by the underlying casManagement uploadTable endpoint (fetched and sent as multipart), which goes beyond the annotations and provides comprehensive behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with clear paragraphs and bullet points, and every sentence provides necessary information. Although somewhat long, it avoids fluff, and the small amount of repetition (e.g., 'read by the server') is used for emphasis rather than redundancy. It is well-organized and efficiently communicates all key aspects.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description appropriately focuses on input and behavior rather than return values. It covers all operational aspects—data sources, file size limits, format handling, and guidance for alternative tools—making it complete for the tool's intended use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant meaning to each parameter beyond the schema descriptions. It explains the file_path semantics in stdio mode, the need for data_format override for URLs without clean suffixes, and clarifies that contains_header_row applies to csv/tsv/Excel. This enrichment makes parameter usage much clearer than the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool uploads a data file into a CAS table and explicitly contrasts it with upload_inline_data for small inline tables. It specifies the two data sources (file_path and URL) and clarifies that data is read server-side, making the tool's purpose unambiguous and distinct from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance by recommending upload_inline_data for small inline tables and offering handling advice for unsupported parquet formats. It also mentions file size limits (MAX_UPLOAD_BYTES) and the need for data_format override in specific cases, giving clear conditions for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![sas-mcp-server MCP server](https://glama.ai/mcp/servers/sassoftware/sas-mcp-server/badges/card.svg)](https://glama.ai/mcp/servers/sassoftware/sas-mcp-server)

Score Badge

Copy to your README.md:

[![sas-mcp-server MCP server](https://glama.ai/mcp/servers/sassoftware/sas-mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/sassoftware/sas-mcp-server)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/sassoftware/sas-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server