Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Rockfish MCP Server start training a TabGAN model on the 'sales' dataset"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A Model Context Protocol server that provides tools to interact with the Rockfish AI platform for synthetic data generation, dataset management, and ML workflow orchestration.
Rockfish API — databases, worker sets, workflows, models, projects, datasets, organizations (22+ tools)
SDK (Synthetic Data Generation)
obtain_train_config — generate training configuration with automatic column type detection
update_train_config — modify training hyperparameters or field classifications
start_training_workflow — start TabGAN training workflow
get_workflow_logs — stream workflow logs with configurable level and timeout
get_trained_model_id — extract trained model ID from completed workflow
start_generation_workflow — start generation workflow from trained model
obtain_synthetic_dataset_id — extract generated dataset ID from completed workflow
plot_distribution — generate distribution plots comparing real and synthetic data
get_marginal_distribution_score — calculate similarity score between real and synthetic data
Manta (Analytics & Scenarios) — requires MANTA_API_URL
discover_schema — discover dataset schema
generate_test_suite — generate test suites
execute_query / execute_nl_query — run SQL or natural language queries
inject_scenario — inject test scenarios into datasets
rockfish-mcp is not currently published to PyPI, so install it from source (below) and run it
with uv run --directory /path/to/rockfish-mcp rockfish-mcp. The uvx rockfish-mcp and
pip install rockfish-mcp forms only work once the package is published to an index.
git clone https://github.com/Rockfish-Data/rockfish-mcp.git
cd rockfish-mcpInstall dependencies (exact versions from uv.lock, including dev tools):
uv sync
source .venv/bin/activate # optional; or prefix commands with `uv run`For a runtime-only install (no dev tools), use uv sync --no-dev.
Set up environment variables:
cp .env.example .env
# Edit .env and add your Rockfish API keyRelated MCP server: RAGFlow MCP Server
Create a .env file with your Rockfish API credentials:
ROCKFISH_API_KEY=your_api_key_here
ROCKFISH_API_URL=https://api.rockfish.aiOptional settings:
ROCKFISH_ORGANIZATION_ID=your_organization_id_here
ROCKFISH_PROJECT_ID=your_project_id_here
MANTA_API_URL=https://manta.rockfish.aiAdd to your claude_desktop_config.json:
rockfish-mcp is not published to PyPI, so uvx rockfish-mcp fails with
rockfish-mcp was not found in the package registry. Run it from a clone instead. Use
uv run --directory rather than uvx --from, so the locked dependencies and the Rockfish SDK
find-links index from pyproject.toml are honored. Use the absolute path to uv
(which uv), since Claude Desktop does not inherit your shell PATH.
{
"mcpServers": {
"rockfish": {
"command": "/opt/homebrew/bin/uv",
"args": ["run", "--directory", "/path/to/rockfish-mcp", "rockfish-mcp"],
"env": {
"ROCKFISH_API_KEY": "your_api_key_here",
"ROCKFISH_API_URL": "https://api.rockfish.ai"
}
}
}
}The first launch builds the environment and can take ~30 seconds; later launches are fast.
{
"mcpServers": {
"rockfish": {
"command": "uvx",
"args": ["rockfish-mcp"],
"env": {
"ROCKFISH_API_KEY": "your_api_key_here",
"ROCKFISH_API_URL": "https://api.rockfish.ai"
}
}
}
}{
"mcpServers": {
"rockfish": {
"command": "python",
"args": ["-m", "rockfish_mcp.server"],
"env": {
"ROCKFISH_API_KEY": "your_api_key_here",
"ROCKFISH_API_URL": "https://api.rockfish.ai"
}
}
}
}{
"mcpServers": {
"rockfish": {
"command": "docker",
"args": ["run", "-i", "--rm", "-e", "ROCKFISH_API_KEY", "-e", "ROCKFISH_API_URL", "rockfish-mcp"],
"env": {
"ROCKFISH_API_KEY": "your_api_key_here",
"ROCKFISH_API_URL": "https://api.rockfish.ai"
}
}
}
}You can use the MCP inspector to debug the server from a source checkout:
npx @modelcontextprotocol/inspector uv run --directory path/to/rockfish-mcp rockfish-mcpOr run the venv's Python directly (after uv sync):
cd path/to/rockfish-mcp
npx @modelcontextprotocol/inspector .venv/bin/python -m rockfish_mcp.serverIf the server fails to start in Claude Desktop, check ~/Library/Logs/Claude/mcp-server-rockfish.log.
Install with dev dependencies (the default for uv sync):
uv syncRun tools with uv run (for example uv run pytest) or activate .venv first.
isort src/rockfish_mcp/ && black src/rockfish_mcp/Unit tests (no credentials required):
pytest tests/test_manta_client.py tests/test_manta_tools.pyIntegration tests (requires .env with real credentials):
pytest tests/
pytest --env=.env.prod.local --html=report.html --self-contained-htmlWe encourage contributions to help expand and improve rockfish-mcp. Whether you want to add new tools, enhance existing functionality, or improve documentation, your input is valuable.
Fork the repository
Create a feature branch
Make your changes
Format your code with isort and black
Add tests if applicable
Submit a pull request
For examples of other MCP servers and implementation patterns, see: https://github.com/modelcontextprotocol/servers
rockfish-mcp is licensed under the MIT License. This means you are free to use, modify, and distribute the software, subject to the terms and conditions of the MIT License. For more details, please see the LICENSE file in the project repository.
Create a new database
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Database name | |
| description | No | Database description |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden, yet it discloses nothing beyond the implied write. It does not state permission/auth requirements, whether names must be unique, what happens on a duplicate name, whether the operation is reversible, or what the response contains.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single, front-loaded sentence with no filler or redundancy. The problem is under-specification rather than bloat, and the little content present is well structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description should at minimum cover naming constraints and side effects. It leaves the agent without enough context to call the tool confidently in an ambiguous sibling set.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters ('name' and 'description') are already documented in the schema. The description adds no format, uniqueness, or length constraints beyond that, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description restates the tool name almost verbatim ('Create a new database' vs create_database) with no added scope, constraints, or distinguishing detail. It conveys the verb and resource but is effectively a tautology, and it does nothing to separate itself from siblings like create_dataset, create_project, or create_workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisite information, and no mention of alternatives or when not to use this tool. An agent gets no help deciding between create_database and the many other create_* siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create a new dataset
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Dataset name | |
| description | No | Dataset description |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It signals a write/create operation but says nothing about required permissions, whether creation is idempotent, what happens on duplicate names, or what the call returns. For a mutation tool with zero annotation coverage this is a substantial gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single short, front-loaded sentence with no wasted words, but its brevity reflects under-specification rather than tight writing. There is no structure to speak of because there is only one clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations, no output schema, and no parameter guidance in the description, an agent is left guessing about side effects, error behavior, and return values. The description is not complete enough to support confident invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and both parameters (name, description) are documented in the schema itself, so the baseline is 3. The description adds no extra meaning, such as name uniqueness constraints or length limits, beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (Create) and resource (dataset), so the operation is unambiguous. However, it is essentially the tool name expanded into a sentence and adds nothing that would distinguish it from sibling creators such as create_database, create_project, or create_workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives, no mention of prerequisites (e.g., an active project context required by get_active_project), and no note about what to do if a dataset with the same name already exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create a new project
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Project name | |
| description | No | Project description |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only repeats the action already implied by the name, with no information about idempotency, duplicate-name behavior, required permissions, or what happens after creation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short and front-loaded, making it easy to parse. The word 'new' is slightly redundant since 'create' already implies newness, but otherwise the phrasing is compact and unfussy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema and no behavioral context. The description does not mention return values, failure modes, name constraints, or how this tool relates to list_projects and other siblings. For a creation tool, this is minimally adequate but leaves important operational details unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes both parameters with 100% coverage, so the schema carries the semantic weight. The description adds no additional meaning about the parameters, which earns the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: 'Create a new project' tells the agent exactly what the tool does. However, it essentially restates the tool name and does not add scope or caveats that would differentiate it from sibling create_* tools beyond the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the verb 'create': the tool is for creating a project. There is no explicit guidance about when to prefer this over listing or using existing projects, nor any mention of prerequisites or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create a new worker set
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Worker set name | |
| worker_count | Yes | Number of workers |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden, yet it says nothing about side effects, permission requirements, or whether the name must be unique. 'Create' implies a mutation, but nothing about reversibility, returned identifiers, or post-conditions is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single four-word sentence with no filler; it is correctly front-loaded. However, the brevity reflects under-specification rather than deliberate economy, so it is adequate but not exemplary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description should at least convey what a worker set is and what happens on success. Instead it relies entirely on the schema, leaving an agent without prerequisite or post-creation context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters ('name' and 'worker_count') are already documented in the schema. The description adds no format, constraint, or uniqueness information beyond that, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Create') and resource ('worker set'), which is enough to distinguish it from the read/delete siblings like get_worker_set and delete_worker_set. It stops short of the 5 level because it does not articulate scope or contrast with any alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no prerequisites, and no mention of the informational siblings (list_worker_sets, get_worker_set). The agent must infer that this is the mutation path for creating a worker set.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create and run a new workflow
| Name | Required | Description | Default |
|---|---|---|---|
| jobs | Yes | Array of jobs that make up the workflow | |
| metadata | Yes | Workflow metadata |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden. It does disclose that creation also triggers execution ("and run"), but says nothing about permissions required, whether execution is asynchronous, idempotency, failure/rollback behavior, or resource consumption — significant gaps for a mutation that starts jobs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single front-loaded sentence with no wasted words, which is good structurally. But the brevity is under-specification rather than effective conciseness for a tool with deeply nested required parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool takes nested objects (jobs with config, input dependencies, worker names/versions, plus metadata) and has no output schema to fall back on. Against that complexity, the description supplies no guidance on constructing the workflow graph or what happens after execution starts, so it is not complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the nested jobs and metadata structures are already documented in the schema, giving a baseline of 3. The description adds no further meaning about the required job aliases, dependency wiring via input, or label syntax beyond what the schema states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb+resource ("Create and run a new workflow"), so an agent knows this produces a workflow and kicks off execution. However it offers no differentiation from siblings such as update_workflow, start_training_workflow, or start_generation_workflow, which also create-and-run workloads.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance is provided. The description never explains how this differs from start_training_workflow or start_generation_workflow, which appear to be specialized variants of the same operation, leaving the agent to guess which entry point to pick.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Delete a database
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Database ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. The verb "delete" implies a destructive, likely irreversible mutation, but the description does not confirm irreversibility, required permissions, or side effects (e.g., whether dependent resources are removed). This is a significant gap for an unannotated destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence that is front-loaded and wastes no words. Its brevity is appropriate to the format, though the missing content is a specification problem rather than a conciseness one.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, unannotated tool with no output schema and no safety metadata, the description leaves out everything an agent would want before invoking it: irreversibility, authorization needs, and side effects. It is not complete enough given the operation's risk profile.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is one parameter and schema description coverage is 100%, so the schema fully documents "id". The description adds no meaning beyond the schema, which merits the baseline 3 for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Delete a database" states a concrete verb (delete) and resource (database), so an agent knows the operation. However, it is essentially a restatement of the tool name and offers no scope detail (soft vs hard delete, cascading behavior) that would separate it from the many other delete_* siblings (delete_dataset, delete_model, delete_worker_set).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use, when-not-to-use, or prerequisite guidance. For a destructive tool sitting alongside delete_dataset and delete_model, an agent gets no help on choosing this one over a sibling or on any required preconditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Delete a dataset
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Dataset ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears the full behavioral burden, yet it only implies a destructive operation. It does not state whether deletion is permanent or reversible, whether it cascades to dependent objects like model bindings or bookmarks, or what permissions are required.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The sentence is short and front-loaded with no wasted words. It is not verbose, but its brevity reflects under-specification rather than disciplined conciseness for a destructive operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, no-annotation tool with no output schema, the description omits everything an agent needs beyond the bare action: irreversibility, side effects on related resources, and error conditions. It is not complete enough to call this tool safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single 'id' parameter is documented as 'Dataset ID' in the schema, so the baseline is 3. The description adds no meaning beyond this, such as where to obtain the ID or acceptable formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states an unambiguous verb (Delete) and resource (dataset), so an agent knows the operation. However, it does nothing to distinguish this tool from its many delete siblings (delete_model, delete_database, delete_worker_set) and adds no scope or constraint detail beyond the tool name itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no mention of alternatives such as update_dataset or soft-delete paths. The agent is left to infer usage entirely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Delete a model
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Model ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden, and it discloses nothing: not whether deletion is permanent or recoverable, whether the model must be inactive/untrained, what permissions are required, or what happens to dependent workflows. For a destructive operation this is a serious gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The single sentence is short but that brevity reflects under-specification rather than efficiency. There is no wasted clause, but there is also no information the agent could not get from the tool name alone.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an irreversible mutation tool with zero annotation coverage and no output schema, the description should at minimum flag destructiveness and any preconditions. It supplies none of that, leaving the agent unable to assess risk before calling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter with 100% schema description coverage, so the schema fully documents the 'id' field. The description adds no meaning beyond it, which is the expected baseline when the schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Delete a model'), which is enough to separate it from delete_dataset and delete_database among the siblings. However, it is essentially a restatement of the tool name with no added scope, constraints, or differentiation from other delete operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no prerequisites, and no mention of alternative tools (e.g., list_models or get_model to confirm the target first). Usage is only implied by the verb.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Delete a worker set
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Worker set ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure, and it offers almost nothing. It does not state that deletion is irreversible, whether it cascades to associated workflows/actions, what permissions are required, or whether confirmation is needed. "Delete" implies destruction but no further context is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with zero waste, and the verb and resource are front-loaded. The brevity is efficient, though it borders on under-specification rather than disciplined conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive mutation tool with no annotations and no output schema, the description should at minimum flag irreversibility or side effects. Given the siblings include many delete operations and related worker-set tools, the complete absence of such context leaves the agent under-informed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single required `id` parameter, which the schema already documents as "Worker set ID." The description adds no syntax, format, or sourcing guidance, so the baseline 3 applies when the schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ("Delete") and resource ("worker set"), so the action is unambiguous. However, it is essentially a restatement of the tool name with no attempt to distinguish it from sibling delete tools such as delete_model, delete_dataset, or delete_database, nor from its counterpart create_worker_set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives or prerequisites (e.g., whether a worker set must be empty or stopped before deletion, or what to use instead). The agent must infer everything from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute a SQL query against Rockfish datasets and return results in CSV format
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | The SQL query to execute. REQUIRED SQL SYNTAX: Use dataset ID wrapped in double quotes as table name (e.g., "rWBroVVFla7I0Fv9nBZXh") and wrap column names in backticks (e.g., `column_name`). Example: SELECT `timestamp`, `value` FROM "rWBroVVFla7I0Fv9nBZXh" WHERE `status` = 'active' | |
| project_id | No | Optional project ID to execute the query in |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It discloses only the CSV output format; it never states whether queries are read-only, whether DDL/DML (INSERT/DELETE/DROP) is permitted, permission/auth requirements, row or time limits, or cost implications for a raw SQL execution surface.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero waste. It is efficient, though arguably too terse for a powerful SQL execution tool with no annotations to back it up.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description's mention of CSV return format usefully covers the result shape, and the schema handles SQL syntax. However, for a raw SQL tool with no annotations, it omits critical operational context (read-only vs mutating, limits, required dataset discovery step).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the query parameter carries detailed syntax guidance (dataset ID in double quotes, columns in backticks) that the thin description does not repeat. Baseline 3 is appropriate since the schema does the heavy lifting and the description adds no parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Execute) and resource (SQL query against Rockfish datasets) plus the return format (CSV), which an agent can distinguish from sibling dataset CRUD tools. It doesn't explicitly name an alternative, but no other sibling executes SQL, so the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to prefer this over the structured dataset tools (e.g., get_dataset, get_dataset_schema, or list_datasets) and no prerequisite noted, such as needing a dataset ID discovered from get_dataset_schema. Usage is only implied by the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get active organization
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure, and it discloses nothing beyond the name. It does not say what the tool returns, whether it can fail or return empty, or whether any auth context is required.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded phrase with no wasted words, which is structurally fine. However, at three words it is minimal to the point of under-specification rather than genuine conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a simple parameterless getter with no output schema, so complexity is low, but the description still leaves the agent with no idea of the return shape or failure behavior. Even a one-line note on what "active organization" means would close the gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to document. Per the baseline for parameterless tools, a 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The phrase "Get active organization" is a bare restatement of the tool name with no additional specificity. It does not distinguish this tool from nearby siblings such as list_organizations or get_active_project, so an agent must guess at the distinction from the name alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this versus list_organizations or get_active_project, nor any mention of prerequisites or the no-active-organization case. Usage is only weakly implied by the word "active".
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get active project
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden, and it discloses nothing: not what 'active' means (session context? default? current org selection?), not whether it can fail when no project is active, and not what shape the result takes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is short and front-loaded, but the brevity is under-specification rather than economy: the single phrase merely echoes the tool name and earns no additional place in the agent's decision process.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, no output schema, and no parameters, the description is the only source of context — and it omits the one thing that matters here, namely the definition of 'active project' and its failure/precondition behavior. It is insufficient for a zero-argument contextual getter surrounded by overlapping siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so per the baseline there is nothing for the description to explain about inputs. No penalty applies here, though there is also no opportunity to add value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a verb ('Get') and a resource ('active project'), which is better than a bare restatement of the name. However, it does not distinguish itself from the sibling tool get_project (or from get_active_organization), leaving the agent to guess what 'active' means or how this differs from a plain project fetch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus get_project, get_active_organization, or list_projects. The only hint is the word 'active' in the name itself, which forces the agent to infer the routing condition rather than being told it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get a specific database by ID
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Database ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. 'Get' implies a read, but there is no statement about permissions, error behavior, or return characteristics. This is a minimal read operation, but the description does not actively disclose behavioral traits beyond the verb.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no wasted words. It is appropriately sized for a simple lookup tool and makes its point immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter lookup with no output schema and no annotations, the description is minimally sufficient to select and invoke the tool. However, it omits usage context and any indication of what is returned, leaving some gaps that a fuller description could address.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single 'id' parameter, so the schema already documents that 'Database ID' is expected. The description's 'by ID' adds no meaning beyond the schema, which matches the baseline of 3 when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb ('Get') and resource ('database') and scopes it to a single record via 'by ID', which distinguishes it from list_databases and the create/update/delete siblings. It stops short of explicitly naming an alternative or sibling, so a 4 is appropriate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The only usage cue is the implicit 'by ID' condition, which tells the agent that an ID is required but does not explain when to choose this tool over list_databases or get_dataset. No alternatives or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get a specific dataset by ID
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Dataset ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden and does not meet it. It does not say whether the call is read-only, what happens on an unknown ID, whether auth or dataset permissions are required, or whether the returned dataset is the full object or a summary.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence, front-loaded with the verb and resource, with no filler. It is efficient, though its brevity shades into under-specification rather than pure conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a low-complexity single-parameter getter with no output schema and no nested objects, the description is minimally viable. However, absent annotations it leaves the safety profile and error/return behavior undocumented, so it is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single id parameter, which the schema itself documents as 'Dataset ID'. The description's 'by ID' adds no format, prefix, UUID-vs-slug, or validity information beyond that, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: retrieve a single dataset identified by ID. That implicitly separates it from list_datasets and from get_dataset_schema, but it never names or contrasts those siblings explicitly, so the differentiation is left to inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus list_datasets, get_dataset_schema, or get_dataset_query_bookmarks. No prerequisites, no exclusions, no context beyond the bare operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Retrieve the query bookmarks for a dataset
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | Dataset ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only implies a read operation via 'Retrieve' but omits permissions, rate limits, return format, pagination, or whether bookmarks are user-specific, providing minimal transparency beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the verb and with no wasted words. It is appropriately sized for a simple retrieval tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with no annotations and no output schema, the description is minimally adequate but lacks context about what bookmarks are returned (e.g., format, owner scope). With no output schema to explain return values, the description should ideally provide more detail to be fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the single dataset_id parameter is fully documented in the schema. The description adds no meaning beyond 'for a dataset,' so the baseline of 3 applies when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Retrieve') and resource ('query bookmarks') with scope ('for a dataset'), making the tool's function immediately clear. It does not differentiate from sibling tools like get_dataset or execute_sql_query, but the resource is distinct enough that an agent can identify it without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, prerequisites, or context. The description only states what it retrieves, leaving the agent to infer usage without any when-not conditions or routing hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the schema for a dataset
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Dataset ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It implies a read operation via "Get" but says nothing about what the schema response contains (column names, types, constraints), whether errors occur for unknown IDs, or any permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler. It is appropriately sized for a one-parameter read tool, though it is terse to the point of omitting useful context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description is responsible for conveying what is returned, but it never explains what the dataset schema contains or its format. Combined with the absence of annotations for a read tool, the definition leaves notable gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single "id" parameter, so the schema already documents it. The description adds no format, source, or lookup detail beyond what the schema provides, making the baseline of 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ("Get the schema for a dataset"), which is clearly distinct from get_dataset, update_dataset, or list_datasets. It is clear but makes no explicit reference to its siblings, so an agent must infer the distinction between fetching a dataset's data versus its schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to call this rather than get_dataset or get_dataset_query_bookmarks, and no stated prerequisites. The purpose is inferable from the name alone, but the description adds nothing about context or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Calculate marginal distribution score between two datasets (real vs synthetic). Lower score means more similar distributions.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_ids | Yes | Exactly 2 dataset IDs: [real_dataset_id, synthetic_dataset_id] |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full disclosure burden. It usefully discloses the metric's semantics (lower score = more similar distributions), but says nothing about whether the call is read-only, how expensive the computation is, or whether it requires a completed training/generation job to exist first. Adequate but with real gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no filler. The core action is front-loaded and the interpretation of the result follows immediately, so every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter compute tool with no output schema, the description supplies the one thing an agent actually needs about the return value: the direction of the metric. It stops short of stating the score's range/scale or the cost of the computation, but it is close to complete for this level of complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single parameter is fully documented in the schema, including the required ordering [real_dataset_id, synthetic_dataset_id]. The description adds no ordering or format detail beyond that, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (calculate) and resource (marginal distribution score) plus the operand (two datasets, real vs synthetic). The meaning of the output is also stated, which lets an agent understand the result without opening a schema. It does not, however, distinguish itself from the nearby plot_distribution sibling, which is also a distribution-comparison tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use, prerequisites, or exclusions are given. The sibling plot_distribution is a plausible alternative for comparing real vs synthetic distributions, and the description never says when to pick this score over that plot. Usage must be inferred entirely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get a specific model by ID
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Model ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It implies a safe read via 'Get', but says nothing about error behavior for a missing/invalid ID, permissions required, or what the returned model object contains — significant gaps for a tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with no filler, and the key qualifier ('by ID') is front-loaded. It is efficient, though it is arguably terse to the point of under-specification rather than maximally useful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read tool with no output schema and full schema coverage, the minimum is met. Still missing are error behavior on unknown IDs and any hint of what a 'model' is here, which would help an agent call it correctly in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single 'id' parameter, so the schema already documents the input fully. The description's 'by ID' merely restates the schema, adding no format, source, or validation detail beyond it; baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (Get) and resource (model) with a clear retrieval-by-ID scope, so the basic purpose is unambiguous. However, it does not differentiate itself from siblings like list_models, get_trained_model_id, or get_model's related delete_model/upload_model, leaving the agent to infer which one fits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives such as list_models (enumerate) or get_trained_model_id (resolve a trained artifact). The phrase 'by ID' weakly implies the caller must already have an ID, but this is inference rather than stated guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get a specific project by ID
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Project ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only says 'get', implying a read operation. It does not disclose error handling (e.g., if project not found), authentication requirements, or return behavior, leaving the agent uninformed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. While concise, it could be more structured by front-loading key information like the return type.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema, no annotations), the description is adequate but lacks details about return values or error responses that would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the 'id' parameter described as 'Project ID'. The description adds no further meaning beyond the schema, which is acceptable for a single required parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (get) and resource (project) and the required parameter (ID). It distinguishes itself from sibling tools like get_projects (plural) and create_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get_projects, create_project, or update_project. The description lacks context about prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract the trained model ID from a completed training workflow. Only works on COMPLETED or FINALIZED workflows.
| Name | Required | Description | Default |
|---|---|---|---|
| workflow_id | Yes | Training workflow ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full behavioral burden. It usefully discloses a hard constraint about workflow states (an error/eligibility condition), but says nothing about what happens on failure, permission requirements, or output form.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with zero filler; the core action is front-loaded and the constraint follows immediately. Nothing is redundant or padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter extraction tool with no output schema and no annotations, the description covers the action and the key eligibility constraint. The only gap is the absence of failure/error behavior detail, which is minor at this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is a single parameter with 100% schema description coverage, so the schema already documents 'workflow_id'. The description adds no format, source, or lookup detail beyond what the schema provides, making the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Extract the trained model ID from a completed training workflow.' This clearly distinguishes it from siblings like list_models or get_model, which deal with model objects rather than the ID produced by a training workflow. It stops short of explicitly naming those siblings, so it is clear but not maximally differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description supplies a precondition ('Only works on COMPLETED or FINALIZED workflows'), which tells the agent when the call will succeed. However, it never names an alternative tool or explains what to do when the workflow is not in one of those states, leaving usage to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get a specific worker set by ID
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Worker set ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not state that this is a read-only operation, whether permissions are required, or what happens when the ID is missing or invalid. Only the bare retrieval action is conveyed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with zero redundancy. It efficiently conveys the core action and lookup key without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read tool with full schema coverage, the description is minimally adequate. However, with no annotations and no output schema, it omits useful context such as return shape, error behavior, or how it relates to sibling tools that retrieve related worker set data.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the single 'id' parameter is already documented in the schema as 'Worker set ID'. The description adds only the phrase 'by ID', which aligns with the schema but does not add format, constraints, or examples beyond what is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Get') and resource ('worker set'), and clarifies the lookup key ('by ID'). It distinguishes the tool from list_worker_sets implicitly by specifying a single item rather than a list, but it does not name or compare against siblings like list_worker_sets or get_worker_set_actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance about when to use this tool versus alternatives such as list_worker_sets or get_worker_set_actions. The requirement of an ID is implied by 'by ID' and the schema, but there is no stated context, prerequisite, or exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List the actions for a worker-set
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Worker set ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. The word 'List' weakly implies a read-only operation, but it says nothing about permissions, whether the result is large or paginated, or what an 'action' represents, leaving a real disclosure gap for even a simple read tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence that is front-loaded and free of filler. It is efficient, though this brevity is partly under-specification rather than disciplined concision.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read tool with a fully documented schema and no output schema, the description is minimally adequate. It is missing the routing information an agent needs to pick it over list_available_actions and any hint of what the returned actions look like.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter is fully documented in the schema ('Worker set ID'), so coverage is 100% and the baseline of 3 applies. The description adds no meaning beyond the schema, such as how to obtain a worker-set ID.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a clear verb (List) and resource (actions for a worker-set), so an agent understands the operation. However, it does nothing to distinguish this from the sibling list_available_actions, leaving ambiguity about which action-listing tool applies to a worker-set versus globally.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus get_worker_set, list_available_actions, or list_worker_sets. No prerequisites, no exclusions, no alternative routing is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get a specific workflow by ID
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Workflow ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. 'Get' implies a read, but the description does not state read-only safety, error behavior for missing IDs, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero waste. It states the action, resource, and key immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read tool with full schema coverage, the description is minimally adequate but omits where the ID comes from and what the returned workflow contains, especially since no output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the id parameter is documented as 'Workflow ID'. The description adds no format, source, or validation details beyond the schema, so the baseline 3 for high coverage applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a precise verb (Get), resource (workflow), and lookup key (ID), clearly distinguishing a single-resource fetch from list_workflows or mutation siblings. It does not explicitly name an alternative or scope, so it falls short of the top tier.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no mention of alternatives such as list_workflows for discovering IDs, and no prerequisites. The agent must infer usage entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Stream logs from a workflow execution with configurable log level and collection timeout. Collects logs for specified duration (default 10s), useful for monitoring workflow progress and debugging. Call repeatedly to continue collecting logs from running workflows.
| Name | Required | Description | Default |
|---|---|---|---|
| timeout | No | Collection duration in seconds. Default: 10 | |
| log_level | No | Log level filter. Default: INFO | |
| workflow_id | Yes | Workflow ID to get logs from |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does well: it discloses the streaming/polling nature, the default 10s collection window, and the repeated-call pattern needed to tail a running workflow. It omits auth requirements or any truncation/pagination behavior, keeping it from a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core action and no wasted preamble. Minor redundancy between "configurable log level and collection timeout" and the following sentence on collection duration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description must convey return behavior; it explains that logs are streamed/collected over a duration, which is the key framing an agent needs. Adequate for a read-only logging tool, with only minor gaps around log volume or format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all three parameters and the enum. The description restates "configurable log level and collection timeout" and the 10s default without adding syntax or behavior beyond the schema, matching the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ("Stream logs from a workflow execution") that is clearly distinct from the get_workflow sibling which retrieves the workflow definition. Purpose is unambiguous, though it does not name any sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear usage context ("useful for monitoring workflow progress and debugging") and actionable guidance ("Call repeatedly to continue collecting logs from running workflows"). No explicit when-not or named alternatives, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List actions (or workers) from all worker-sets
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It does not say this is a read-only operation, whether results are paginated, or what an 'action' record contains. For a bulk listing across all worker-sets, the potential result size and shape go undisclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with the resource and scope front-loaded and no wasted words. The parenthetical alias slightly muddies an otherwise crisp statement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a no-parameter list tool with no output schema, the description is minimally adequate. It omits what an 'action' is, why it is equated with 'workers', and whether the result spans the whole tenant, leaving noticeable gaps for an agent deciding whether this is the right call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so per the rubric the baseline is 4. The description adds terminology context by equating 'actions' with 'workers', which is mildly useful but does not concern parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('List') plus resource ('actions (or workers)') and scope ('from all worker-sets'). The scope phrase meaningfully separates it from the sibling get_worker_set_actions, which presumably targets a single set. The parenthetical 'or workers' introduces a naming ambiguity but does not obscure the core purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit statement of when to use this versus get_worker_set_actions or get_worker_set. The phrase 'from all worker-sets' implicitly signals the cross-set scope, which is the only usage cue present, so guidance is inferable but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all databases
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states 'List all databases' without disclosing behavioral traits such as read-only nature, authentication requirements, or return format. This is insufficient for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant information. It is appropriately concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no parameters, no annotations, and no output schema. The description is minimal but adequate for a simple list operation. However, it does not specify the return format or any additional context like pagination, so it is not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters with 100% schema description coverage (vacuously). The description adds no parameter meaning, but per rubric the baseline is 3 when coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all databases' uses a specific verb and resource, clearly indicating the action of listing all databases. It distinguishes from siblings like get_database (specific database) and create_database (creation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The context is clear from the name and description: use this to list all databases. However, it does not explicitly mention when not to use it or alternatives (e.g., get_database for a single database), so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all datasets
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. The word "all" hints that the call is unfiltered read-only, but nothing is said about pagination, ordering, volume limits, or return shape.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three words, zero waste, front-loaded with the verb. It is terse but not padded, which is appropriate for a no-argument list call.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (no params, no output schema, no annotations), so the bar is low, but with no output schema the description could still note what it returns or whether results are paginated. It is minimally adequate rather than complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so per the rubric the baseline is 4. The description adds nothing about parameters, but there is nothing to add.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description pairs a clear verb ("List") with a specific resource ("datasets"), so the operation is unambiguous. It does not differentiate itself from similarly-named siblings such as list_databases or list_projects, but the resource noun does that implicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this tool versus alternatives like get_dataset or list_databases, and no prerequisites or context. The agent must infer usage entirely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all models
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It does not say whether the result is paginated, ordered, filtered by project/organization scope, or what fields each model contains, and there is no output schema to fill that gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler, which is proportionate for a zero-parameter list tool. It is arguably under-specified rather than verbose, but nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-param, no-output-schema tool, the description is minimally sufficient to select it, but the agent cannot anticipate scope (all projects? active organization?) or return shape. A sentence on scoping or result contents would close the gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate and the baseline of 4 applies. Schema coverage of 100% on an empty property set is consistent with the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (models) with the scope 'all', so the agent knows this is an unfiltered enumeration. It does not distinguish itself from nearby siblings like get_model or list_datasets, but the verb+resource pairing is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this instead of get_model, or when listing is appropriate versus fetching a single model. The word 'all' faintly implies a bulk enumeration, but there is no explicit when/when-not or alternative routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all organizations
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. 'List' implies a read-only, non-mutating call, but nothing is said about auth requirements, whether the result is paginated, or whether it covers every org the caller can see. For a zero-annotation tool this is a thin disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single four-word sentence, front-loaded and free of filler. It is efficient, though the extreme terseness leaves it doing no more than the minimum.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read tool with no output schema, the description is technically sufficient to invoke the call. However, it omits whether results are truncated or paginated and how they relate to the singular get_active_organization sibling, leaving modest gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so per the rubric the baseline is 4. There is no parameter surface for the description to explain, and schema coverage is 100%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (organizations) with the scope modifier 'all', so the operation is unambiguous. It does not explicitly differentiate itself from the sibling get_active_organization, though the 'all' vs 'active' contrast is arguably self-evident.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no prerequisites, and no mention of the sibling get_active_organization as the alternative for fetching a single/current org. The agent must infer the context entirely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all projects
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description implies a read operation ('List'), which is appropriate, but does not disclose any limitations (e.g., pagination, filtering) or confirm safety. Adequate for a simple list tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise single sentence with no waste. However, it is so brief that it borders on under-specification, lacking any structured detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, no output schema), the description fully captures its purpose. No additional information is required for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has no parameters, and coverage is 100%. Description adds no value beyond schema, but schema already fully describes the lack of parameters. Baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'List' and resource 'all projects'. Distinguishes from siblings by resource type, but lacks additional context that could differentiate from other list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get_project or list_servers. The description is too minimal to provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all worker groups
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. 'List all' implies a read-only enumeration, but it says nothing about pagination, filtering, result size, or whether the list is scoped to the active project/organization — notable given siblings like get_active_project and get_active_organization.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no filler or redundancy. It is efficient, though the brevity is partly under-specification rather than deliberate economy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, no output schema, and no parameters, very little is required beyond naming the resource. Still, the unresolved group-vs-set ambiguity against four sibling tools leaves a real gap for an agent deciding which listing tool to call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a no-parameter tool is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a verb (List) and a resource (worker groups), so the basic operation is clear. However, the sibling set contains list_worker_sets, get_worker_set, create_worker_set and delete_worker_set, and the description never clarifies how a 'worker group' relates to a 'worker set' — the exact distinction an agent needs to choose between them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no mention of prerequisites, and no reference to the alternative list_worker_sets or the get_* siblings. The agent must infer entirely from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all worker sets
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden, yet it discloses nothing behavioral: no pagination, no scoping (organization/project), no return shape, no permission requirements. It only implies a read operation via 'List'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single short clause with no filler, which is structurally fine, but the brevity reflects under-specification rather than disciplined concision — there is nothing front-loaded beyond the bare verb.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Even for a simple zero-param list tool with no output schema and no annotations, the description omits whether results are scoped to an organization/project or paginated, which are real questions for a 'list all' tool. It is not adequate on its own.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to clarify; baseline 4 applies. The description does not mislead about inputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description is essentially the tool name expanded into a sentence: 'list_worker_sets' → 'List all worker sets'. It adds the scope word 'all' but no distinguishing information relative to siblings like get_worker_set or list_worker_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no mention of alternatives (get_worker_set, list_worker_groups, list_worker_sets' relationship to projects/organizations), and no preconditions. The agent must infer selection from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all workflows
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden, yet it discloses nothing about scope (organization/project), pagination, ordering, or permissions. For a read-only listing tool the risk is low, but the description adds no behavioral context beyond the verb.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with zero waste and the action front-loaded. It is terse to the point of being under-specified, but there is no padding or redundancy to penalize.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter list tool with no output schema, little explanation is strictly required. However, it omits the scoping of the listing (global vs. active project) and any indication of result breadth, leaving the agent to guess.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so per the baseline there are no parameter semantics to document. The description correctly implies a parameterless enumeration.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a clear verb (List) and resource (workflows), so an agent immediately knows the operation. It does not differentiate from siblings like get_workflow, list_worker_sets, or create_workflow, though 'all' implies a bulk enumeration rather than a single fetch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this versus get_workflow or list_worker_sets, no stated prerequisites, and no exclusions. The agent must infer usage context entirely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract the synthetic dataset ID from a completed generation workflow. Only works on COMPLETED or FINALIZED workflows.
| Name | Required | Description | Default |
|---|---|---|---|
| generation_workflow_id | Yes | Generation workflow ID from start_generation_workflow |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations and no output schema, the description carries the full burden. It usefully discloses the state precondition (COMPLETED/FINALIZED), but says nothing about failure behavior on non-completed workflows, whether the ID persists, or what the return looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, zero waste, with the action front-loaded and the constraint immediately following. Nothing extraneous.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter extraction tool with full schema coverage, the description is nearly sufficient. The main gap is the absence of any statement about the returned ID, which the missing output schema leaves to the agent's inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single parameter is already documented as coming from start_generation_workflow. The description adds no syntax, format, or sourcing detail beyond the schema, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Extract') and resource ('synthetic dataset ID') tied to a generation workflow, so an agent can distinguish it from siblings like get_trained_model_id or start_generation_workflow. It does not explicitly name the sibling it complements, so it falls just short of full differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear precondition: it 'only works on COMPLETED or FINALIZED workflows,' which tells the agent when the tool is valid. It does not name an alternative or say what happens if the workflow is still running, but the triggering condition is explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate training configuration for synthetic data models. Returns a cached train_config_id for use in start_training_workflow. Supports TabGAN (rf_tab_gan) and TimeGAN (rf_time_gan).
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | Dataset ID to generate config for | |
| model_type | Yes | Model type: 'rf_tab_gan' for tabular data or 'rf_time_gan' for time-series data |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are supplied, so the description carries the full behavioral burden. It does disclose that the config is cached and returned as an ID, which hints at repeat-call behavior, but it never states side effects, whether regenerating overwrites an existing config, cost/latency, or permission requirements for a generation operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with purpose before the return value and supported types. No filler, no restatement of the tool name; every sentence carries information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-required-parameter tool with no output schema, the description usefully substitutes for return documentation by naming the train_config_id and its consumer. It is nearly complete; only error/prerequisite behavior and regeneration semantics are unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and model_type carries an enum with per-value guidance, so the schema already documents both parameters fully. The description only restates the model type identifiers, adding no syntax, constraints, or edge cases beyond what structured fields provide — baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource ('Generate training configuration for synthetic data models') with the return contract stated ('cached train_config_id') and the downstream consumer named ('start_training_workflow'). Supported model types are enumerated, so the agent knows exactly what this tool produces and where it fits relative to the workflow siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly positions this as the step that feeds start_training_workflow, which tells the agent when to reach for it. It stops short of explicit exclusions (e.g., when to use update_train_config instead of regenerating) or prerequisites about the dataset_id needing to exist, so it is clear context without routing rules.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate distribution plot comparing datasets for a specific column. Returns base64-encoded PNG image. Automatically chooses bar plot (categorical) or KDE plot (numerical).
| Name | Required | Description | Default |
|---|---|---|---|
| column_name | Yes | Column name to plot distribution for | |
| dataset_ids | Yes | List of dataset IDs to compare (e.g., [real_dataset_id, synthetic_dataset_id]) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does meaningful work: it discloses the return format (base64-encoded PNG) and the automatic selection logic between bar (categorical) and KDE (numerical), letting the agent predict the output. It omits any statement of side effects, permissions, or size/time limits, keeping it out of 5 territory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, no filler, and the core purpose is front-loaded ahead of the return format and plot-type behavior. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly fills that gap by stating the return type (base64 PNG), and the required parameters are covered by the schema. It is nearly complete, lacking only edge-case behavior such as what happens with unsupported column types or large datasets.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters (column_name, dataset_ids) are already documented in the schema, including the real-vs-synthetic example. The description adds only the notion of cross-dataset comparison, which is marginal beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Generate distribution plot') plus scope ('comparing datasets for a specific column'), so an agent knows exactly what comes back. It stops short of distinguishing itself from the related sibling get_marginal_distribution_score (plot vs. score), which would have made it a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'comparing datasets for a specific column', which signals the comparison scenario, but there is no explicit when-to-use, when-not-to-use, or named alternative (e.g., use get_marginal_distribution_score for a numeric score instead of a visual).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Start a workflow to generate synthetic data from a trained TabGAN model.
| Name | Required | Description | Default |
|---|---|---|---|
| model_id | Yes | Trained model ID from get_trained_model_id |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, so the description carries the full burden. It doesn't disclose that this is an asynchronous workflow start, what it returns (a workflow ID, a job handle?), how to poll progress (get_workflow_logs?), or any cost/rate implications — all critical for a workflow-launcher.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and purpose. No filler; every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A workflow-launching mutation with no annotations and no output schema leaves the agent without the async/return/polling context it needs. The existence of get_workflow_logs and obtain_synthetic_dataset_id siblings hints at the lifecycle, but the description doesn't connect them.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single required model_id is documented with a pointer to get_trained_model_id. The description adds only the 'trained TabGAN model' qualifier, so it earns the baseline rather than more.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource+outcome: 'Start a workflow to generate synthetic data from a trained TabGAN model.' The 'generation' framing distinguishes it from the sibling start_training_workflow, though it doesn't name that sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance. It doesn't state prerequisites (e.g., that a model must already be trained and its ID obtained via get_trained_model_id) or how it relates to obtain_synthetic_dataset_id, which likely consumes this workflow's output.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Build and start a training workflow using a cached config. Automatically detects model type from train_config structure. Use this after obtaining train_config_id from obtain_train_config.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | Dataset ID to train on | |
| train_config_id | Yes | Config ID from obtain_train_config (retrieves cached config) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and does disclose two traits: it performs a mutation (build and start) and auto-detects model type from train_config structure. It omits permissions/auth needs, whether the workflow runs asynchronously, failure behavior, and reversibility, so it is only partially transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core action and scope, with no wasted wording. The auto-detection and prerequisite notes each add information rather than padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description should explain what the agent gets back (e.g., a workflow ID) and whether execution is asynchronous. It covers the input-side prerequisite but leaves the outcome and lifecycle underspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters are already documented in the schema, establishing a baseline of 3. The description adds modest value by clarifying that train_config is a cached config whose structure drives model-type detection, but no format or syntax detail beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific compound verb (build and start) and resource (training workflow), which clearly separates it from the sibling start_generation_workflow and the obtain_* tools. It does not explicitly name and contrast the closest sibling, but the verb+resource is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives the prerequisite sequence: 'Use this after obtaining train_config_id from obtain_train_config', naming the dependency tool. However, it offers no exclusions or conditions for when NOT to use it versus starting a generation workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update a database
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Database ID | |
| name | No | Database name | |
| description | No | Database description |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full disclosure burden, and it discloses nothing. For a mutation tool, it fails to say whether the update is partial or full, whether omitted fields are preserved or cleared, whether permissions are required, or whether the change is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The single four-word phrase is not verbose, but it is under-specified rather than concise—it omits information the agent needs. Conciseness credit requires that the remaining content earn its place, and here there is essentially no content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
A mutation tool with three parameters, no annotations, and no output schema demands a description that covers behavior and effects. The description is completely inadequate for that burden, leaving the agent unable to predict what updating a database will do.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and each of the three parameters (id, name, description) is documented in the schema with a type and label. The description adds no syntax, format, or constraint details beyond the schema, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description restates the tool name almost verbatim ("Update a database" for update_database), adding no scope, no field information, and no differentiation from siblings like update_project, update_dataset, or update_workflow. An agent learns only the obvious verb+resource pairing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives, no mention of prerequisites, and no exclusion conditions. The description offers nothing an agent could use to select this tool over the many other update_* siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update a dataset
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Dataset ID | |
| name | No | Dataset name | |
| description | No | Dataset description |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full behavioral burden and discloses nothing. It does not say whether this is a partial or full replacement update, what happens to omitted fields, whether permissions are required, or whether the change is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The single sentence is short but that is under-specification rather than conciseness. There is no wasted wording, yet the sentence does not earn its place by conveying any information an agent could act on.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The three parameters are simple and fully documented in the schema, so basic invocation is feasible, but critical semantics for a mutation tool (patch vs replace behavior, effect of omitted fields) are absent from both the description and the annotations, leaving the definition thin.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (id, name, description all documented in the schema), so the baseline is 3 even though the description contributes no parameter meaning. It does not clarify, for example, that omitting name or description leaves those fields unchanged.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a dataset' merely restates the tool name in sentence form, adding no scope, no list of updatable fields, and no distinction from siblings like update_project, update_workflow, or update_database. An agent learns nothing beyond what the name already conveys.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus siblings such as create_dataset or delete_dataset. No prerequisites, no mention of whether it is safe to call repeatedly, and no stated alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update a project
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Project ID | |
| name | No | Project name | |
| description | No | Project description |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure, yet it provides none. It does not describe whether the update is partial or full replacement, whether it requires specific permissions, whether changes are reversible, or what the response looks like. This is a significant gap for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short and front-loaded, but it is under-specified rather than appropriately concise. For a three-parameter update tool, a three-word sentence fails to cover necessary scope and behavior, so it is not sized correctly for its job.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a mutation tool with no annotations, no output schema, and a simple three-parameter schema, the description is incomplete. It omits usage guidance and behavioral context that an agent needs, though the schema does handle parameter details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters (id, name, description) with clear descriptions. The description adds no additional meaning beyond the schema, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a project' is a near-verbatim restatement of the tool name, adding no specificity about what can be updated or how it differs from sibling tools like create_project or get_project. The schema provides the actual fields, but the description itself does not earn its place beyond the name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no indication of prerequisites, and no mention of alternatives such as create_project or get_project. The agent receives zero routing help for choosing this tool over its siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update training configuration. Modify model hyperparameters (epochs, batch_size, etc.) and/or encoder field classifications (categorical, continuous, ignore). See https://docs.rockfish.ai/model-train.html for config structure.
| Name | Required | Description | Default |
|---|---|---|---|
| updates | Yes | Updates to apply: model_config for hyperparameters, encoder_config for field types | |
| train_config_id | Yes | Config ID from obtain_train_config |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. 'Update' implies mutation, but the description never states whether it requires specific permissions, whether updates merge with or replace existing config, whether changes are reversible, or whether a running training workflow is affected. For a mutation tool with zero annotation coverage this is a real gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the core action and following with the modifiable areas and a doc link. Little waste, though the URL is a pointer rather than content an agent can act on directly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given rich nested schema coverage and no output schema, the description covers the essentials of what is being modified and points to external docs. It is nonetheless thin on behavioral context (permissions, merge semantics, effect on active jobs) for a mutation tool with no annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the nested model_config/encoder_config structure and train_config_id are already documented inline, meeting the baseline. The description's examples (epochs, batch_size, categorical/continuous/ignore) largely restate the schema rather than adding format or constraint detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb+resource (update training configuration) and enumerates the two modifiable areas: model hyperparameters and encoder field classifications. An agent can distinguish it from obtain_train_config. However, it does not explicitly name the sibling relationship or contrast with start_training_workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no when-to-use, when-not-to-use, or alternative guidance. The only hint of workflow context ('Config ID from obtain_train_config') lives in the schema, not the description, so the agent gets no routing signal from the prose itself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update a workflow
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Workflow ID | |
| name | No | Workflow name | |
| definition | No | Workflow definition |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure, and it says almost nothing. 'Update' implies an in-place mutation but the description does not state permission requirements, whether the change is reversible, or whether omitted fields (name, definition) are preserved or cleared. Only the bare implication of mutation is conveyed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single three-word phrase with no wasted words, but this is under-specification rather than true conciseness. There is nothing front-loaded because there is nothing substantive to load.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with a nested-object parameter, no annotations, and no output schema, the description should at minimum explain what 'definition' represents and whether partial updates are supported. None of that context is present, leaving the agent to guess at semantics the structured fields alone cannot supply.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters (id, name, definition). The description adds no syntax, format, or partial-update semantics beyond that, so the baseline 3 for high-coverage schemas applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a workflow' merely restates the tool name (update_workflow) and title with no additional specificity about what is being changed. It gives a verb and resource but no fields or scope, so an agent gains nothing it didn't already have from the name. It does not distinguish this tool from adjacent updaters like update_project or update_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no stated when-to-use, no prerequisites, and no reference to alternative tools such as get_workflow or create_workflow. The agent must infer that this is the mutation counterpart to those siblings entirely on its own. No guidance is provided, though nothing is actively misleading.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Upload a new model
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Model name | |
| metadata | No | Model metadata |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden and provides none of it: no statement of required auth or permissions, no duplicate-name handling, no indication of what happens on failure, and no hint that 'metadata' is a free-form nested object. A mutation tool with zero annotation coverage needs far more than a four-word fragment.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The sentence is front-loaded and wastes no words, but it is under-specified rather than genuinely concise. Nothing needs trimming; the problem is that there is too little content, not too much.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations, no output schema, and a nested object parameter ('metadata') whose internal shape is undefined, the description leaves the agent without enough to invoke it confidently. Return behavior and the semantics of the nested object are unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (name and metadata both documented), so the baseline is 3. The description adds no additional meaning — it does not explain what 'metadata' should contain or whether 'name' must be unique — but the schema already covers the declared fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Upload a new model" gives a recognizable verb and resource, so an agent can tell it creates a model resource rather than reading one. However, it makes no attempt to distinguish itself from siblings like list_models, get_model, or delete_model beyond the verb, and 'upload' is ambiguous about whether bytes, an archive, or a config is being submitted.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no mention of alternatives such as create vs. update paths. The agent must infer from the name alone that this is the creation entry point for models.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0create_databasecreate_datasetcreate_projectcreate_worker_setcreate_workflowdelete_databasedelete_datasetdelete_modeldelete_worker_setexecute_sql_queryget_active_organizationget_active_projectget_databaseget_datasetget_dataset_query_bookmarksget_dataset_schemaget_marginal_distribution_scoreget_modelget_projectget_trained_model_idget_worker_setget_worker_set_actionsget_workflowget_workflow_logslist_available_actionslist_databaseslist_datasetslist_modelslist_organizationslist_projectslist_worker_groupslist_worker_setslist_workflowsobtain_synthetic_dataset_idobtain_train_configplot_distributionstart_generation_workflowstart_training_workflowupdate_databaseupdate_datasetupdate_projectupdate_train_configupdate_workflowupload_modelScored across 44 tools
Most tools target a distinct resource and action (e.g., create_dataset, get_workflow, start_training_workflow). A few listing tools overlap in scope, such as list_available_actions versus get_worker_set_actions, and list_worker_groups versus list_worker_sets, but descriptions generally clarify the differences.
Nearly all tools follow a consistent snake_case verb_noun pattern. Minor deviations include using obtain instead of get for extraction tools and upload_model instead of create_model, but the overall convention remains predictable.
With 44 tools, the surface is heavy for the apparent synthetic-data platform scope. While the domain is broad, many operations could likely be consolidated or grouped, and 25+ tools is generally too many for an agent to navigate efficiently.
Core synthetic data lifecycle is well covered: dataset CRUD, SQL query, model upload, training config, training workflow, generation workflow, and evaluation plots. Minor gaps exist, such as missing delete_project, delete_workflow, and update_model, but agents can work around these.
Run any Relay model (image, video, audio, LLM), poll tasks, upload files and search Relay's docs.
List datasets, schemas, run APL queries, and use prompts for exploration, anomalies, and monitoring.
AI-powered data integration platform. Onboard users and run DPF data workflows.
Manage portable AI agent playbooks, Agent Skills, MCP configurations, personas, and memory.
