Tuner MCP
Tuner MCP is a control plane for model training, dataset preparation, evaluation, and checkpoint management over Tinker/Cookbook, exposing 48 MCP tools.
Dataset operations: search Hugging Face datasets, probe pinned splits, fetch/stage datasets, prepare local or inline data, validate and inspect datasets.
Training: plan and run SFT, DPO, RL, and distillation; start/stop/resume runs; monitor metrics, logs, checkpoints.
Evaluation: run Cookbook benchmarks, compare runs, inspect failures and artifacts.
Model utilities: sample base models/checkpoints, compute logprobs, preview rendering.
Recipe management: list/get official Cookbook recipes, validate and start recipe plans.
Planning: autoplan experiments combining live models, recipes, and HF candidates.
Checkpoint management: list, export, publish/unpublish, set TTL, delete checkpoints.
Session and usage: list/inspect sessions, export traces, get account usage.
Object persistence: list and retrieve saved datasets and plans.
Connects to the Hugging Face Hub to search public datasets by popularity, language, or task, probe their configurations and samples, and fetch pinned (40-character commit SHA) dataset splits into validated, staged training data. Also supports experiment autoplanning by combining Hugging Face candidates that pass schema probing, with gated repos requiring an HF token.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Tuner MCPprepare my dataset for SFT, then run a 30-minute fine-tune"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Tuner MCP
Tuner is an independent, agent-native control plane over the public Tinker SDK and Tinker Cookbook. It exposes safe JSON contracts for dataset validation, sampling, supervised fine-tuning, preference learning, RL, distillation, evaluation, checkpoints, and experiment comparison.
Tuner is an independent open-source project built using the public Tinker SDK and Tinker Cookbook. It is not affiliated with or endorsed by Thinking Machines Lab.
Prefer a visual workflow? Tuner UI is the first-party browser interface and delegates live training to this control plane.
Agent skill
SKILL.md gives Codex and other skill-aware agents the complete
operating guide for Tuner. It routes agents across all 48 MCP tools and 34
reviewed Cookbook recipes, including dataset preparation, bounded training,
monitoring, evaluation, checkpoint management, usage reporting, recovery, and
the rule that live Tuner work must use native MCP tools rather than shell or
direct API substitutes. Detailed tool, recipe, contract, workflow, and runtime
references live under references/.
To make the repository-owned skill available to a local Codex installation on Windows without creating a second copy, link this repository as the skill directory and restart Codex:
New-Item -ItemType Junction `
-Path "$env:USERPROFILE\.codex\skills\tuner-mcp" `
-Target (Resolve-Path .)Related MCP server: Oumi MCP Server
Use cases
These workflows progress from basic model and data operations to advanced, multi-stage experimentation.
Basic
Discover available Tinker models. List supported models and compare size, context length, modality, and availability.
Find Hugging Face datasets. Search by language, domain, task, author, popularity, or tags.
Inspect and validate datasets. Check splits, columns, message structure, preference pairs, samples, and formatting problems.
Prepare reproducible training data. Convert data into conversation, instruction, preference, prompt, or tool-call formats with deduplication, seeded shuffling, and validation splits.
Preview model-ready examples. Inspect the rendered prompt, token count, truncation, and assistant loss mask before training.
Intermediate
Run bounded supervised fine-tuning. Adapt a model for a language, domain, writing style, instruction-following behavior, or company-specific task.
Train a tool-calling or CLI assistant. Teach command generation and structured tool calls using demonstrations and validation data.
Preference-align a model with DPO. Improve response quality, style, safety, or concision using chosen and rejected answers.
Evaluate and compare models. Measure held-out loss, exact or normalized answers, tool-call correctness, and Cookbook benchmarks.
Manage training and checkpoints. Monitor metrics, inspect failures, stop or resume supported runs, manage retention, and export PEFT or merged Hugging Face models.
Advanced
Train autonomous coding and terminal agents. Combine SFT with Code RL or Harbor/TerminalBench and sandboxed execution feedback.
Distill expert models into smaller models. Use one or multiple teachers with on-policy, off-policy, reasoning, prompt, or self-distillation.
Build search and retrieval agents. Train models to decide when to search, construct queries, interpret evidence, and return grounded answers.
Run multi-stage RLHF or continual specialization. Chain supervised, preference, RL, and continual-learning stages while retaining dataset and checkpoint lineage.
Build a closed-loop improvement workflow. Discover data, run a bounded pilot, evaluate it, inspect failures, create corrective data, retrain, compare checkpoints, and export the selected model. Today these steps are composed from MCP tools; fully automatic multi-stage orchestration remains an implementation target.
Quick start
For this Windows workspace, use the Docker setup below. The configured Codex
endpoint is http://127.0.0.1:8765/mcp; run ./scripts/doctor.ps1 to check it.
See the tool workflow and limits and
current implementation status.
The following direct-install commands require Linux or WSL for Cookbook support.
uv sync
$env:TINKER_API_KEY = "..."
uv run tuner --transport stdioThe Cookbook runtime uses CPU PyTorch for model-specific rendering, local tensor
preparation, and recipe-specific losses. Tinker still executes all GPU training
remotely. The upstream tml-renderers package does not publish a Windows
distribution, so run live Cookbook workflows in Docker, Linux, or WSL.
Local Streamable HTTP:
$env:TUNER_AUTH_TOKEN = "<generate-a-long-random-token>"
uv run tuner --transport http --host 127.0.0.1 --port 8000The HTTP MCP endpoint is http://127.0.0.1:8000/mcp and requires
Authorization: Bearer <TUNER_AUTH_TOKEN>. HTTP mode fails closed when the token
is absent. Stdio does not require this transport token.
Docker
Run the authenticated HTTP server with Docker Compose:
$env:TINKER_API_KEY = "..."
$env:HF_TOKEN = "..." # optional; increases Hugging Face download limits
$env:TUNER_AUTH_TOKEN = "<a-random-token-at-least-32-characters-long>"
docker compose -f compose.standalone.yaml up --build -dThe service binds only to 127.0.0.1:8765, persists Tuner state in the
tuner-state volume, and exposes files under ./data as read-only /data.
The standalone Compose file starts its own Redis service and persistent cache.
The workspace-specific compose.yaml can instead use an existing
tuner-test-redis container through the external tuner-mcp-network. Set that
network up once before using that file:
docker network create tuner-mcp-network
docker network connect tuner-mcp-network tuner-test-redisThe workspace-specific service connects to Redis as
redis://tuner-test-redis:6379/0, uses the external tuner-harbor-cache
volume, and keeps Hugging Face model downloads in tuner-model-cache. For an update
to the existing installation, build docker build -t tuner-mcp:candidate ., run
./scripts/deploy.ps1, then ./scripts/doctor.ps1. Deployment checks for active
runs and retains the old image under a timestamped rollback tag. It reads the existing Windows
user credentials without printing them. Never run both servers against the same
queue/state at once. A client MCP restart may be needed to attach updated tools.
The image installs the pinned official tinker-cookbook[all] distribution and
the CPU-only PyTorch wheel. Some recipes still require their own configured
services: Modal for code/Harbor execution, Gemini plus Chroma for Search Tool,
and external environment credentials for Verifiers.
Dataset format
Conversation JSONL contains one object per line:
{"messages":[{"role":"user","content":"What is 2+2?"},{"role":"assistant","content":"4"}]}Paths are restricted to TUNER_ALLOWED_ROOTS (semicolon-separated on Windows,
colon-separated elsewhere). The current directory is the default allowed root.
Finding Hugging Face datasets
dataset_search_hf searches public Hub datasets by popularity without leaving Tuner:
{"request": {"query": "tool calling function calling", "sort": "likes", "limit": 5}}Single-term queries (function-calling, terminal-bench, shell) rank best.
Each hit returns hf_repo, pinned sha, likes, downloads, and the gated flag —
pass hf_repo + sha to dataset_probe_hf before dataset_fetch_hf. Search is read-only and
free; Hub outages surface as retryable DATASET_ERROR, never fabricated entries.
Fetching Hugging Face datasets
dataset_probe_hf first inspects configuration names and samples a pinned split
to report compatible mappings. dataset_fetch_hf then streams that pinned
public dataset split into a staged,
validated dataset_id usable by every training tool:
{
"request": {
"hf_repo": "teknium/OpenHermes-2.5",
"hf_revision": "<40-character commit SHA>",
"hf_split": "train",
"output_type": "conversation_jsonl",
"max_records": 1000,
"selection_strategy": "reservoir",
"scan_records": 10000,
"shuffle_seed": 30
}
}Raw xLAM datasets are also supported without generating replacement records. Use
the mapping returned by dataset_probe_hf; for the original Salesforce dataset
the request has this shape:
{
"request": {
"hf_repo": "Salesforce/xlam-function-calling-60k",
"hf_revision": "<40-character commit SHA from dataset_search_hf>",
"hf_split": "train",
"output_type": "conversation_jsonl",
"user_field": "query",
"tool_calls_field": "answers",
"tools_field": "tools",
"max_records": 1000,
"validation_records": 100,
"shuffle_seed": 30
}
}Rules: the revision must be the full 40-character commit SHA (no branch names,
so a re-resolve can never silently change the data); trust_remote_code is
always off; rows are mapped to messages (ShareGPT conversations convert
automatically, JSON-encoded message strings are parsed), preference triples, or prompt text depending on
output_type; instruction-style rows map via user_field/assistant_field
(e.g. instruction→user, cmd→assistant); unmapped rows fail fast naming
the record index and available fields. The original Salesforce xLAM repository is
gated: accept its Hub terms and configure HF_TOKEN in the server environment.
When Hub data is requested, Tuner reports access or schema blockers instead of
silently replacing it with synthetic inline records. dataset_prepare
accepts the same hf_repo/hf_revision pair for one-step local-or-HF staging.
Use selection_strategy: "reservoir" with a fixed shuffle_seed to select a
deterministic representative sample from scan_records; the default head
strategy reads only max_records. Prepared manifests report prompt diversity
without discarding Albanian or other non-Latin scripts, plus distributions for
available intent, category, domain, and language metadata.
dataset_prepare also accepts up to 1,000 inline_records, so an MCP client can
persist synthetic or hand-authored tool-use demonstrations without creating a
server-local file:
{
"request": {
"inline_records": [
{
"messages": [
{"role": "user", "content": "Find the latest run."},
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"type": "function",
"function": {"name": "training_list", "arguments": "{\"limit\":1}"}
}
]
}
],
"tools": [
{
"type": "function",
"function": {
"name": "training_list",
"description": "List runs",
"parameters": {"type": "object", "properties": {"limit": {"type": "integer"}}}
}
}
]
}
],
"output_type": "conversation_jsonl",
"validation_records": 0
}
}Stored provenance contains the inline count and content hash rather than a second copy of the full inline payload.
Preference datasets can map common Hub field names explicitly with
preference_prompt_field, chosen_field, and rejected_field. The probe
returns these selectors when it recognizes fields such as
instruction/chosen_response/rejected_response; staged output always uses
canonical prompt or messages, chosen, and rejected keys. Full-conversation
chosen/rejected rows that repeat the shared prompt are normalized to assistant-only
completions. Set invalid_record_policy to skip to omit invalid rows and record
removal counts in the transform manifest; the default error policy fails preparation.
Conversation rows may carry a top-level tools list. Tuner preserves tool
schemas, tool call IDs, assistant calls with null content, tool results, and
Cookbook message fields such as name and unparsed_tool_calls.
Selecting an experiment automatically
experiment_autoplan combines live Tinker capabilities, Cookbook renderer
metadata, and Hugging Face candidates that pass schema probing. It selects a
trainable model and returns an exact dataset_fetch_request, a staged recipe
sequence, and blockers. Multi-configuration datasets include hf_config:
{
"request": {
"objective": "Train a CLI assistant that makes reliable tool calls",
"task": "terminal_agent",
"constraints": {"max_model_params_billions": 10}
}
}Use recipes_list and recipe_get to discover every reviewed official
Cookbook recipe. recipe_plan validates exact upstream config fields for a
selected recipe and recipe_start runs that immutable plan through the same
Redis-backed execution lifecycle as the typed SFT/DPO/RL/distillation tools.
Safety and cost
Live sampling, evaluation, and training require TINKER_API_KEY and may incur
charges. Normal tests never make live Tinker calls. Training limits are controlled by
TUNER_MAX_TRAINING_STEPS, TUNER_MAX_DATASET_BYTES, TUNER_MAX_SAMPLES, and
TUNER_MAX_GENERATION_TOKENS.
TUNER_MAX_TOTAL_GENERATION_TOKENS caps aggregate generated tokens per operation,
and TUNER_MAX_PROMPT_BYTES bounds direct sampling/logprob request payloads.
TUNER_TASK_URL defaults to memory:// for the local MVP. A remote deployment
can point it at a supported Redis/Valkey URL and run the corresponding worker model.
Run metadata is persisted in SQLite under the state directory. Existing JSON run
records are imported without deleting the originals. Idempotency keys are scoped
to the operation: the same request replays its run, and changed content returns
IDEMPOTENCY_CONFLICT. Admission and execution claims are transactional across
processes sharing that database.
Training and evaluation use the shared controller for concurrency limits,
cancellation records and heartbeats. Startup marks stale active runs as needing
reconciliation; it does not replay training. Cancellation stops local orchestration;
already submitted remote work may continue. training_resume starts a new SFT
attempt from saved optimizer/epoch/batch state; train_dpo, train_rl and
train_distill are exposed. Typed SFT and DPO have completed bounded live verification;
typed RL and distillation have also completed bounded live verification. The current
public-release evidence and remaining clean-machine gate are tracked in
docs/RELEASE_TEST_MATRIX.md.
training_metrics returns recent metrics before completion. Pass cursor=0 to
page complete JSONL records and use the returned byte cursor for the next page.
training_logs lists nested artifacts and inlines only metrics/checkpoint JSONL by default;
pass artifact_path and cursor to read bounded chunks. TUNER_MAX_ARTIFACT_BYTES caps each read. Training admission also
checks TUNER_MAX_BATCH_SIZE, TUNER_MAX_INPUT_TOKENS and
TUNER_MAX_CONCURRENT_RUNS.
Typed plans report method-specific natural steps after holdouts, whether each important hyperparameter was explicit or resolved from Cookbook defaults, the Cookbook learning-rate recommendation, and dataset concentration warnings. Distillation currently accepts exactly one epoch because its typed adapters make one dataset pass.
Capabilities distinguish credential presence from verified connectivity.
live_available is unknown (null) until a live capability check succeeds.
Evaluation is marked as mutating because it creates run records and artifacts.
Custom evaluation supports exact, normalized exact, structured tool-call, and
domain-rubric scoring. Rubric scoring requires a separate Tinker judge model and
stores bounded judge evidence. compare_runs returns inconclusive instead of a
winner for truncated, incompatible, undersized, below-threshold, or statistically
weak results.
train_sft/dpo/rl/distill and evaluate accept background=true to submit
through Docket and return a run_id promptly; native task clients get progress
the same way. checkpoint_export queues work by default and promptly returns an
export_id; poll it with training_get and stop it with training_stop. Pass
background=false only when the archive or conversion is known to fit the client
deadline. It returns a signed archive URL (tinker_archive) or builds a PEFT adapter /
merged HF model (peft/hf_merged). PEFT preserves the adapter produced by Tinker's
remote checkpoint exporter and does not download the base model. Merged-HF runs through
Cookbook on Linux/WSL and needs the full base model; the persistent model cache avoids
downloading that model again on later merges.
MCP client configuration
{
"mcpServers": {
"tuner": {
"command": "uv",
"args": ["--directory", "C:/path/to/tuner", "run", "tuner"]
}
}
}Run uv run tuner-discover to refresh generated/tinker_api_manifest.json after
updating either upstream repository.
Available Tools
48 toolsbenchmarks_listBenchmarks ListARead-only
List benchmark names discovered from installed Cookbook source.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint=true annotation already tells the agent this is a safe, non-mutating operation. The description adds that results are scoped to benchmarks 'discovered from installed Cookbook source', which is useful context about the data origin, but it does not mention pagination, ordering, or whether an empty result is possible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single efficient sentence with the core action and scope front-loaded. Nothing is wasted and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation, and there are no parameters to document. The description conveys what is listed and where it comes from, leaving only minor gaps such as result ordering or scope limits.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so no parameter semantics are required; the baseline for a no-param tool is 4. The description correctly implies no input filtering is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (benchmark names) with a qualifier about origin (discovered from installed Cookbook source). This clearly distinguishes it from the sibling recipes_list, which enumerates recipes rather than benchmarks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the name and description (use it to enumerate available benchmarks), but there is no explicit when-to-use, when-not-to-use, or named alternative. An agent can infer the intent but receives no routing guidance versus siblings like recipes_list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
capabilities_getCapabilities GetBRead-onlyIdempotent
Describe Tuner's tool surface and optionally fetch live Tinker server capabilities.
| Name | Required | Description | Default |
|---|---|---|---|
| live | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so safety is covered. The description adds one useful behavioral fact — that a live Tinker server round-trip can happen — but says nothing about latency, failure modes when the server is unreachable, or the difference between the cached and live views.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the primary action ('describe Tuner's tool surface') comes before the optional secondary behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return-value details are unnecessary, and the one-sentence scope is enough to call the tool correctly. Slight gap: nothing tells the agent what 'tool surface' encompasses or when the live view is worth requesting.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the schema gives no meaning for `live`; the description is the only source and it does imply that the flag triggers a live Tinker fetch. However, it never states the type, default, or cost of that live call, so it only partially compensates for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: describes Tuner's tool surface and can fetch live Tinker capabilities. It is clearly distinct from all siblings (recipes, training, datasets, checkpoints), though it never explicitly says how it differs from a generic listing tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when an agent should call this versus other discovery tools such as models_list or recipes_list, and no stated condition for when to set the live flag beyond the word 'optionally'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkpoint_deleteCheckpoint DeleteBDestructive
Permanently delete the checkpoint at this exact Tinker path.
| Name | Required | Description | Default |
|---|---|---|---|
| tinker_path | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and destructiveHint=true, so safety is covered structurally. The description reinforces irreversibility with 'Permanently' and adds the exact-path constraint, but says nothing about permissions, whether deletion cascades, or recoverability.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no wasted words; the destructive nature and the path constraint both land immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, and the tool is a simple one-parameter mutation whose safety profile is carried by annotations. Only the path format and irreversibility details are thin.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% for the single tinker_path parameter, so the description must compensate. 'At this exact Tinker path' implies exact matching (no globbing or prefix resolution), which is useful, but the expected path format is not explained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Permanently delete the checkpoint') with the scoping qualifier 'at this exact Tinker path'. This distinguishes it from siblings like checkpoint_set_ttl or checkpoint_export, though it never names them explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this versus checkpoint_set_ttl, checkpoint_unpublish, or other checkpoint lifecycle tools. The agent must infer the destructive-delete use case purely from the verb.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkpoint_exportCheckpoint ExportBIdempotent
Queue a checkpoint export, or wait when background=false.
| Name | Required | Description | Default |
|---|---|---|---|
| format | Yes | ||
| background | No | ||
| base_model | No | ||
| tinker_path | Yes | ||
| idempotency_key | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations include readOnlyHint=false and idempotentHint=true, but the description does not add behavioral context beyond the queue/wait behavior. There's no mention of required permissions, potential long-running nature, side effects, or what happens on failure. With annotations present, the bar is lower, but the description adds minimal value beyond what's already known.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with moderate information, but it is under-specified rather than concise. It front-loads the core purpose but omits essential parameter guidance, making it feel incomplete rather than efficient. It could be more helpful with a brief list of key parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 params, 1 enum, background option, idempotency key) and zero schema descriptions, the description is far from complete. It does not cover return values (output schema exists but the description doesn't reference it), error conditions, or format-specific requirements. An agent would likely need to inspect the output schema or guess parameter semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any parameter meaning. It doesn't clarify the relationship between format and base_model, the purpose of idempotency_key, or the effect of background. With 5 parameters, 2 required, and no schema descriptions, the description leaves the agent without semantic help for most parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Queue') and resource ('checkpoint export'), and adds an operational nuance ('or wait when background=false') that distinguishes it from simple queue tools. However, it does not explicitly differentiate from sibling checkpoint tools like checkpoint_publish or checkpoint_list, leaving some ambiguity about export vs publish.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'or wait when background=false' gives a hint about usage modes, but it does not provide clear guidance on when to use this tool over alternatives like checkpoint_publish or training_get. There's no explicit 'when to use' or 'when not to use' context, so the agent must infer usage from the name and sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkpoint_getCheckpoint GetARead-only
Get checkpoint weight metadata using an exact tinker:// path.
| Name | Required | Description | Default |
|---|---|---|---|
| tinker_path | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already tells the agent this is a safe read. The description adds one useful behavioral detail beyond that: it returns "weight metadata" rather than weights or a payload, which matters for a checkpoint tool. It says nothing about error behavior for a bad path or about the returned fields, leaving meaningful gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler. Every clause (metadata, exact path, URI scheme) carries information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return-value detail is not required, and the read-only annotation covers safety. What is missing is the linkage to checkpoint_list as the source of a valid path and any hint about failure modes, leaving the definition thinner than the surrounding checkpoint_* namespace warrants.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single parameter is documented only as "string". The description partially compensates by specifying the tinker:// URI scheme and the requirement that the path be exact, but it does not say where such a path comes from or what an invalid one does.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ("Get checkpoint weight metadata") that separates it from sibling checkpoint_list (enumeration) and checkpoint_export/delete/publish (mutations). However, it never names those siblings, so the agent must infer the boundary itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase "using an exact tinker:// path" implies this is a point-lookup that requires a known path, which indirectly suggests checkpoint_list is the way to discover one. No explicit when-to-use or when-not-to-use statement is given, so the guidance remains implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkpoint_listCheckpoint ListBRead-only
List training and sampler checkpoints for a Tinker training run.
| Name | Required | Description | Default |
|---|---|---|---|
| training_run_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already tells the agent this is a safe read. The description adds the useful scope detail that both training and sampler checkpoints are returned, but says nothing about ordering, pagination, or freshness of results beyond what annotations cover.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence that wastes no words. It could optionally carry a scope or ordering note, but nothing is redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation, and readOnlyHint covers the safety profile. For a one-parameter read tool this is nearly sufficient, with only the sibling disambiguation and pagination behavior left unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (training_run_id) at 0% schema coverage, but the name is self-explanatory and the description's 'for a Tinker training run' ties it to the run context. No format or acquisition hints are added beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('List') and resource ('training and sampler checkpoints') scoped to a training run, which is more precise than the title. It does not explicitly contrast with siblings like checkpoint_get or the mutating checkpoint_* tools, but the list/retrieve distinction is inferable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this versus checkpoint_get or the many other checkpoint_* siblings. The agent must infer usage from the name alone; no prerequisites or exclusions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkpoint_publishCheckpoint PublishBIdempotent
Make this exact checkpoint publicly accessible on Tinker.
| Name | Required | Description | Default |
|---|---|---|---|
| tinker_path | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=false and idempotentHint=true, so the mutation and repeat-safety profile is already covered. The description adds the meaningful consequence that the checkpoint becomes 'publicly accessible', which the annotations do not convey. It stops short of saying whether this replaces an existing published version, what audience 'public' means, or what authorization is required.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with the visibility outcome front-loaded and no filler. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema exists, so return values need no explanation, and the mutation/idempotency profile is covered by annotations. However, the description omits how the tinker_path is sourced and what 'publicly accessible' entails, leaving the agent with an under-specified invocation for a state-changing operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%: tinker_path is an undescribed string. The phrase 'this exact checkpoint' loosely implies tinker_path identifies the checkpoint, but the description never explains the path format, where it comes from, or how to obtain it. With one undocumented parameter, the description does not compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (make publicly accessible) and resource (checkpoint) with a scope qualifier ('this exact checkpoint'), so the agent knows it operates on one identified checkpoint rather than a list. It implies the inverse of checkpoint_unpublish but never names or contrasts with that sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this over alternatives, no prerequisites (e.g. checkpoint must already exist or be owned by the caller), and no mention of checkpoint_unpublish or checkpoint_set_ttl. Usage must be inferred entirely from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkpoint_set_ttlCheckpoint Set TtlBIdempotent
Set checkpoint retention; null requests indefinite retention.
| Name | Required | Description | Default |
|---|---|---|---|
| tinker_path | Yes | ||
| ttl_seconds | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and idempotentHint=true, so the mutation and repeat-safety profile is covered. The description adds one genuine behavioral fact beyond the schema: null means indefinite retention. It still omits whether existing retention is overwritten, whether the change is reversible, and any permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with the core action front-loaded and the null semantics appended as a qualifier. No filler, though the extreme brevity borders on under-specification for a two-parameter mutation tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation, and annotations cover safety. However, with two required parameters at 0% schema coverage and no usage context, an agent must guess at the meaning of tinker_path and the practical effect of the call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the full burden, and it only clarifies ttl_seconds' null case. tinker_path is completely unexplained, and the unit/format expectation for ttl_seconds (seconds, per the name only) is never stated in prose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Set checkpoint retention.' An agent can tell this mutates a retention policy rather than reading or deleting a checkpoint. It does not differentiate itself from siblings such as checkpoint_delete or checkpoint_publish, which also mutate checkpoint state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to call this versus alternatives, what preconditions exist (does the checkpoint need to be published? must the caller own it?), or what effect retention has on later operations. The only conditional given is about a parameter value, not about usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkpoint_unpublishCheckpoint UnpublishBIdempotent
Remove public access to this exact checkpoint on Tinker.
| Name | Required | Description | Default |
|---|---|---|---|
| tinker_path | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and idempotentHint=true, so safety and repeatability are covered. The description adds useful specificity that the operation revokes public visibility rather than destroying the checkpoint, but says nothing about permissions, error behavior if the checkpoint was never published, or reversibility.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence with the action front-loaded and no filler. Nothing could be removed without losing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and the idempotency annotation covers repeat invocation. Still, for a mutation tool the definition is thin on preconditions (does it error if the checkpoint is already private?) and on the tinker_path parameter, which is undocumented everywhere.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the single parameter tinker_path is documented nowhere. The description gestures at 'this exact checkpoint' but gives no format, origin, or valid-value guidance for the path, leaving the agent to guess how to construct it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Remove public access ... checkpoint'), which is far more precise than a generic 'unpublish'. It does not, however, distinguish itself from close siblings like checkpoint_delete or checkpoint_set_ttl, so the agent must infer the boundary from the wording alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use / when-not-to-use guidance and no named alternative. The phrase 'this exact checkpoint' hints at scope but never says when an agent should unpublish versus delete, set a TTL, or simply leave the checkpoint private.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_runsCompare RunsCRead-onlyIdempotent
Compare compatible evaluations using an optional explicit metric policy.
| Name | Required | Description | Default |
|---|---|---|---|
| metric | No | ||
| direction | No | ||
| baseline_run_id | Yes | ||
| candidate_run_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the safe-read profile is covered. The description adds some value by noting that evaluations must be 'compatible' and that the metric policy is optional, but it does not explain what compatibility means, what happens if runs are incompatible, or how the optional policy affects behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no padding. It is appropriately sized, though the phrase 'compatible evaluations using an optional explicit metric policy' is somewhat jargon-heavy and could be clearer without adding length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a comparison tool with four parameters at 0% schema description coverage, the description is too thin. The output schema means return values need not be explained, and annotations cover safety, but the description still fails to explain compatibility, the role of the baseline and candidate run IDs, or how the metric policy is applied.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for all four parameters. It only vaguely gestures at an 'optional explicit metric policy,' which might loosely relate to the metric and direction parameters, but it says nothing about baseline_run_id or candidate_run_id and does not clarify the direction enum or metric format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Compare') and an object ('compatible evaluations'), which gives a basic sense of the tool's purpose. However, 'compatible evaluations' is vague and the description never mentions the baseline/candidate run IDs that are central to the operation, so it does not distinguish this tool clearly from siblings like evaluate or evaluation_get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as evaluate, evaluation_get, or evaluation_failures. The phrase 'optional explicit metric policy' hints at a default behavior but does not explain when to supply a policy or when an implicit policy is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compute_logprobsCompute LogprobsBRead-only
Compute prompt token log probabilities for a base model or checkpoint.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds only that the operation targets a base model or checkpoint; it does not describe return behavior, latency, or other operational traits beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single, front-loaded sentence with no wasted words. However, for a tool with a deeply nested input, the brevity underspecifies the callable structure rather than being optimally concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a complex nested schema and many parameters, but the description omits key input semantics such as the renderer, the messages/token_ids distinction, and how model vs checkpoint_path should be supplied. An output schema exists, so return values need not be covered, but the input side is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has a single complex request object with 0% description coverage. The description only vaguely maps to target (base model or checkpoint) and prompt token log probabilities, leaving prompt.messages vs prompt.token_ids, renderer, and nested message structures unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: compute prompt token log probabilities. It also scopes the target as a base model or checkpoint, but it does not explicitly differentiate from siblings such as evaluate or sample.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the purpose, but the description provides no explicit when-to-use guidance, prerequisites, or alternatives. It does not say when to choose this over evaluate, sample, or other inference-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_fetch_hfDataset Fetch HfC
Fetch a pinned Hugging Face split, map rows to a Tuner schema, stage a dataset ID.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The only annotation is readOnlyHint:false, so the description must explain side effects. It says the tool 'stages a dataset ID,' which hints at mutation/persistence, but it does not clarify what staging means, whether it writes to the workspace, or whether it is idempotent. No auth, network, or rate-limit considerations are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one concise sentence with no filler, and the main action is front-loaded. It earns its place, though it is arguably too terse for a tool with a complex nested request object and many configuration options.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity, minimal annotations, and 0% schema description coverage at the top level, the description is incomplete. It does not address the intended workflow context, important configuration policies such as invalid_record_policy and deduplicate, or what consuming the staged dataset ID implies. The output schema helps, but the operational context remains underspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, but it only hints at 'pinned' and 'map rows' without explaining the many field-mapping parameters like user_field, assistant_field, tools_field, output_type, or invalid_record_policy. The high-level pipeline is stated, but parameter-level meaning is largely left to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Fetch') and resource ('pinned Hugging Face split'), then adds transformation and output context ('map rows to a Tuner schema', 'stage a dataset ID'). It is clearly distinguishable from siblings like dataset_search_hf and dataset_probe_hf, though it does not explicitly contrast itself with dataset_prepare.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given for when to use this tool instead of the many sibling dataset tools, especially dataset_search_hf, dataset_probe_hf, or dataset_prepare. There are no exclusions, prerequisites, or alternative selection criteria, so the agent must infer usage from the name and brief description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_inspectDataset InspectCRead-onlyIdempotent
Validate and return a small preview of a local dataset.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset | Yes | ||
| sample_size | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds that a preview is 'small' and that the input is local, but says nothing about what happens on an invalid dataset (error vs partial report), whether the preview is truncated/paginated, or cost/latency for large files.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with the action front-loaded and zero filler. It is appropriately sized, though its brevity is partly under-specification rather than true economy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return-value detail is not required, but with a nested dataset object, low schema coverage, and two near-identical siblings, the description omits everything an agent needs beyond the name: which dataset types are accepted, what a preview contains, and how it differs from dataset_validate or dataset_render_preview.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is effectively 0% for the two top-level parameters: sample_size, its default of 3, and the meaning of each type enum value (conversation_jsonl vs preference_jsonl vs prepared, etc.) are never explained. The description mentions no parameters at all, so it fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb pair and resource: it validates a dataset and returns a preview. That is clearer than the bare name, but it never separates itself from close siblings like dataset_validate (which presumably also validates) or dataset_render_preview (which also previews), so an agent cannot tell which of the three to pick from the description alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use statement, no prerequisites, and no named alternative despite at least two overlapping siblings (dataset_validate, dataset_render_preview). The only implicit guidance is 'local dataset', which also leaves the relationship to dataset_prepare/prepared types unstated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_prepareDataset PrepareB
Stage validated local or pinned Hugging Face data as a persistent dataset ID.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds that data becomes a persistent dataset ID, which is consistent with readOnlyHint=false and clarifies that this is a staging/write operation. However, it does not disclose validation behavior, the meaning of 'validated' or 'pinned', or any side effects beyond persistence. With minimal annotation coverage, more behavioral context would be valuable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no filler, front-loading the action and outcome. It is appropriately sized for a simple summary and does not waste tokens.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema and many nested parameters with descriptions, the tool is complex, and the description leaves major gaps: no alternative routing, no validation semantics, no lifecycle or persistence details. An agent operating from this description alone would struggle to know whether to call dataset_validate first or how to configure the request.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not compensate by explaining the required 'request' parameter or any of the nested fields. Saying 'local or pinned Hugging Face data' hints at dataset and hf_repo, but an agent cannot construct a valid request from the description alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Stage'), a resource type ('validated local or pinned Hugging Face data'), and a concrete outcome ('persistent dataset ID'). It is clear enough to distinguish this from dataset_fetch_hf or dataset_probe_hf, though it does not explicitly contrast with sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use dataset_prepare versus alternatives such as dataset_validate, dataset_inspect, or dataset_fetch_hf. The description implies usage by stating what the tool does, but it provides no conditions, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_probe_hfDataset Probe HfCRead-only
Probe configs and row mappings for a pinned Hugging Face dataset split.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already tells the agent this is a safe read, so the description carries a lower burden, but it adds almost nothing beyond that — it does not say what 'probing' returns, whether it fetches data or only metadata, or any auth/rate-limit context. The word 'pinned' only weakly signals the immutable-revision requirement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a single well-formed sentence with no filler, and the resource is front-loaded. It is efficient, though under-specified rather than overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with a required nested request object of five undocumented fields and no annotation depth, the description is too thin; the existing output schema means return values need not be explained, but the input semantics and purpose distinctions are still missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single parameter is a nested object with five sub-fields (hf_repo, hf_revision, hf_split, hf_config, sample_records) that are entirely undocumented. The description's mention of a 'pinned ... split' only faintly gestures at hf_revision and hf_split and does not compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a verb ('probe') and resource ('configs and row mappings for a Hugging Face dataset split'), so the general domain is inferable. However, 'probe' is vague, and the definition does nothing to separate it from close siblings like dataset_inspect, dataset_validate, or dataset_search_hf.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this tool versus the many dataset siblings (dataset_prepare, dataset_fetch_hf, dataset_search_hf, dataset_validate, dataset_inspect, dataset_render_preview). No prerequisites, no exclusions, no conditions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_render_previewDataset Render PreviewBRead-only
Preview Cookbook rendering and loss masks without submitting training.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| dataset | No | ||
| plan_id | No | ||
| renderer | No | ||
| sample_size | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already tells the agent this is non-mutating, so the description only needs to add context. It does add one useful behavioral fact: no training job is submitted as a side effect. Beyond that it says nothing about cost, latency, or what the preview contains.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the core action and the key constraint both land in the first clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, but with five undocumented parameters and no schema descriptions the agent cannot tell which inputs are required, what 'renderer' selects, or how to choose between plan_id and dataset. The description is far too thin for a tool with this many knobs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across five parameters, so the description carries the full burden and fails it. It never explains model, dataset, plan_id, renderer, or sample_size, nor how plan_id relates to dataset (the only hint anywhere is the nested 'path' description in the schema).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (preview) and a specific resource (Cookbook rendering and loss masks), which is enough to distinguish it from mutation-oriented siblings like train_sft or training_start. It does not, however, differentiate itself from neighboring read/inspect tools such as dataset_inspect or dataset_validate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'without submitting training' implies the intended context: use this to sanity-check data before committing to a training run. That is only implied, not stated as explicit when-to-use guidance, and no alternative tool (dataset_inspect, dataset_validate) is named or excluded.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_search_hfDataset Search HfCRead-only
Search public Hugging Face datasets by popularity; returns repo IDs and SHAs.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=true, so the description carries the burden of behavioral disclosure. It mentions the search is over public datasets and returns repo IDs and SHAs, but says nothing about pagination, result caps, or sorting behavior beyond 'popularity' when the schema allows likes/downloads/recent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single efficient sentence, front-loaded with the verb and resource. No waste, though it is arguably under-specified rather than merely concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need not be explained, but the nested request object has 5 undocumented parameters and there are no annotations covering behavior beyond readOnly. For a search tool with 5 filter parameters, the description is too thin.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% at the top level, and the nested request object's fields (sort, tags, limit, query, author) are mostly undocumented. The description only hints at popularity sorting and return fields, not compensating for the coverage gap on tags, limit, or author.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Search) and resource (public Hugging Face datasets) with scope 'by popularity'. Distinguishes from sibling dataset_fetch_hf/dataset_probe_hf to a degree, though it doesn't name them explicitly to route the agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance. Nothing tells the agent how this differs from dataset_fetch_hf or dataset_probe_hf, which appear adjacent in the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_validateDataset ValidateBRead-onlyIdempotent
Validate a local JSON/JSONL dataset, reporting exact malformed record indexes.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered. The description adds that failures are reported as exact malformed record indexes, which is useful output behavior, but says nothing about handling of mixed-validity files, size limits, or whether it stops at the first error.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence stating action, input, and returned value with no filler. Nothing is redundant with the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return format need not be restated, and the description still usefully flags the malformed-index report. However, for a nested parameter at 0% schema coverage and a six-value enum, the description leaves meaningful ambiguity an agent must resolve elsewhere.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single nested object parameter contains an undocumented enum (conversation_jsonl, preference_jsonl, prompt_jsonl, json, jsonl, prepared). The description's 'JSON/JSONL' hints at two enum values but leaves the other four, and the distinction between 'path' and 'dataset_id', unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific verb (validate) and resource (local JSON/JSONL dataset) and adds the distinguishing outcome, malformed record indexes. It is clearly separable from siblings like dataset_prepare or dataset_inspect, though it does not explicitly name them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'local JSON/JSONL dataset' and the schema's 'prepared' type, suggesting pre-flight validity checking. There is no explicit when-to-use or when-not-to-use guidance, and no routing to alternatives such as dataset_inspect or dataset_prepare.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
evaluateEvaluateC
Run a Cookbook benchmark and persist evaluation artifacts. This spends credits.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes | ||
| background | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotation readOnlyHint=false only signals a write; the description usefully adds that the call spends credits and persists artifacts, which the annotations do not convey. Beyond that it says nothing about runtime, background execution (despite a 'background' parameter), idempotency, or failure/cancellation behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no filler, and the action plus cost warning are front-loaded. Both sentences earn their place, though the total size is arguably thin for a tool with this much configuration surface.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with a deeply nested 2-parameter request object, no parameter descriptions, minimal annotations (readOnlyHint only), and no parameter guidance in prose, the description is far too sparse. An output schema exists, so return values need not be explained, but the invocation contract is effectively undocumented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description explains none of the nested request fields, leaving ambiguous pairs like benchmark vs benchmarks, target.model vs checkpoint_path, idempotency_key, and background undocumented. Defaults and enums in the schema carry some meaning, but the description contributes nothing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Run a Cookbook benchmark') and adds the side effect ('persist evaluation artifacts'), which distinguishes it from the read-side siblings evaluation_get and evaluation_failures. It does not name a sibling explicitly, but the action is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this tool versus benchmarks_list, evaluation_get, evaluation_failures, or the training/experiment siblings. The only qualifier is the cost note, which advises caution but does not route the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
evaluation_failuresEvaluation FailuresBRead-only
Read evaluation trajectories with errors or non-positive rewards.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| offset | No | ||
| evaluation_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already tells the agent this is a safe read. The description's real contribution is defining what counts as a 'failure' (errors or non-positive rewards), which is genuine behavioral context beyond the annotation. It says nothing about pagination behavior despite limit/offset existing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with the resource and the filter front-loaded. Every word earns its place and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described. However, with 0% schema description coverage and no mention of evaluation_id or the limit/offset paging semantics, the definition is thin for a tool an agent must scope correctly on first call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across three parameters, so the description carries the full burden — yet it never mentions evaluation_id, limit, or offset. The required scoping parameter evaluation_id is entirely unexplained, leaving the agent to infer its meaning from the name alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (read) and resource (evaluation trajectories) plus a discriminating filter (errors or non-positive rewards), which separates it from evaluation_get and experiment_rollouts. It stops short of naming those siblings explicitly, so an agent must infer the boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the filter phrase — call this when you want only failing/negative-reward trajectories — but there is no explicit when-to-use, when-not, or named alternative. An agent can guess the intent but gets no routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
evaluation_getEvaluation GetBRead-only
Inspect an evaluation's persistent status, benchmark scores and artifacts.
| Name | Required | Description | Default |
|---|---|---|---|
| evaluation_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already tells the agent this is a safe read. The description adds the useful behavioral fact that the status returned is persistent and that artifacts are included, which goes somewhat beyond the annotation. It does not state whether results are cached, how long they persist, or whether the evaluation must have completed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tightly-packed sentence that front-loads the verb and resource and enumerates the returned artifacts with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values need not be explained. The description covers purpose and scope adequately for a read-only retrieval tool. Missing only usage context and any pointer to the evalu-ation_failures sibling for error detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter, evaluation_id, with 0% schema description coverage and no description detail. An agent must assume evaluation_id is the identifier returned by an evaluate call. Baseline 3 applies for a single-param tool, but the lack of any format or source hint is a minor gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (inspect) and resource (an evaluation), and lists the concrete things retrieved: persistent status, benchmark scores, artifacts. It is clear what the tool returns. It does not name or distinguish itself from sibling retrieval tools like evaluation_failures, though the scope is distinct enough to infer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no mention of prerequisites (e.g., an evaluation must exist), and no routing to alternatives such as evaluation_failures for error details. The agent must infer usage entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
experiment_artifactsExperiment ArtifactsBRead-only
List a run's files or read one artifact using a bounded byte cursor.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | ||
| limit | No | ||
| cursor | No | ||
| offset | No | ||
| run_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds genuinely useful behavioral context by disclosing the "bounded byte cursor" pagination mechanic, but it says nothing about error handling for missing paths, how cursor and offset interact, or response shape expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence that conveys the tool's two modes with zero filler. Well-sized for the information it chooses to deliver.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists and annotations cover read safety, five parameters with 0% schema coverage and two distinct operating modes are not adequately explained by one sentence. An agent can guess at path/cursor but cannot confidently use limit or offset, making the definition incomplete for the tool's actual complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 5 parameters, so the description must carry the burden. It only hints at path (read one artifact) and cursor (bounded byte cursor); run_id, limit, and offset receive no explanation, and the critical relationship between cursor and offset is undefined.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states two specific modes with a clear resource: listing a run's files and reading a single artifact. The verb+resource is unambiguous and the dual-mode framing helps an agent understand the tool's identity, though it does not explicitly contrast itself against siblings like object_get or experiment_rollouts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the phrase "or read one artifact" suggests that supplying a path switches to single-artifact read mode, but there is no explicit when-to-use, when-not-to-use, or named alternative. Adequate but leaves mode selection to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
experiment_autoplanExperiment AutoplanC
Select live models, Cookbook recipes, and pinned HF candidates without training.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint=false the annotation signals this is not a pure read, but the description ('select ... without training') reads like a non-mutating selection, leaving it unclear whether the call persists any artifacts or state. It adds only the 'without training' scope note and omits auth requirements, side effects, or whether the plan is saved.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with the verb first and no filler; every word earns its place. It is efficient, though arguably too terse for a tool of this complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with a nested request schema, enum task types, constraint sub-object, and a non-read-only profile, a single sentence is inadequate. The output schema exists so return values need not be explained, but usage, side effects, and parameter meaning are all missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the single nested `request` object carries many sub-fields (task enum, objective, constraints, hf_* options) with no textual documentation. The description provides no parameter meaning whatsoever, so it fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a concrete verb ('Select') and specific resources ('live models, Cookbook recipes, and pinned HF candidates') plus a discriminating qualifier ('without training'). It is clear what the tool does, though it does not name or contrast itself with close siblings like recipe_plan or training_plan, which also produce selections/plans.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no alternatives named. The sibling set contains several planning-style tools (recipe_plan, training_plan, experiment_artifacts), and the description does nothing to route an agent between them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
experiment_rolloutsExperiment RolloutsCRead-only
Read a bounded page of persisted rollout or evaluation trajectories.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| offset | No | ||
| run_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already establishes that this is a safe read. The description usefully adds that results are 'persisted' (stored, not computed on the fly) and 'bounded' (paginated rather than unbounded), which goes beyond the annotation. It stops short of describing page-size limits, ordering, or consistency behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with no filler and the key scoping concept ('bounded page') front-loaded. It is efficient, though arguably too sparse for a tool with three undocumented parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema is present, so return values need no explanation, and the annotation covers the safety profile. But with zero parameter documentation and no usage context, the definition is insufficient for an agent to know what run_id means or how paging works.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so all three parameters (run_id, limit, offset) are undocumented anywhere. The description's phrase 'bounded page' loosely implies limit/offset semantics but never explains them, and the required run_id is not described at all, leaving the agent to guess what identifier it must supply.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb 'Read' plus the resource 'persisted rollout or evaluation trajectories' is specific and concrete, and 'bounded page' signals pagination. It does not, however, name or distinguish itself from nearby siblings like evaluation_get, training_metrics, or training_logs, which an agent would need to route correctly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no mention of alternative tools for retrieving trajectories or metrics. The agent is left to infer that this is the tool for paging through rollout data rather than any of the ~45 sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
models_listModels ListARead-onlyIdempotent
List Cookbook-known models; live mode returns authoritative server-supported models.
| Name | Required | Description | Default |
|---|---|---|---|
| live | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so safety is covered. The description adds genuine behavioral context beyond that: it explains the default returns a cached/known catalog while live mode hits the server for authoritative results, which signals differing freshness and cost.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One compact sentence, front-loaded with the default behavior and then the mode switch. No filler or restatement of the name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return-value detail is unnecessary, and the tool is a simple single-flag list operation. The description covers both modes adequately, though it does not state whether results are paginated or the shape differs between modes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single 'live' parameter is undocumented in the schema, so the description carries the burden. It does explain what live=true yields (authoritative server-supported models) versus the default, which is meaningful semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('List') and resource ('Cookbook-known models'), and distinguishes two operational modes via the live flag. It does not name or differentiate itself against any sibling, but the resource is distinct enough that confusion is unlikely.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use guidance or named alternative, but the clause 'live mode returns authoritative server-supported models' implies the selection condition for live=true versus the default. Usage is inferable rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
object_getObject GetBRead-only
Retrieve a saved dataset or resolved plan by its persistent ID.
| Name | Required | Description | Default |
|---|---|---|---|
| object_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safe-read profile is covered. The description adds that the ID is 'persistent', implying stable references across sessions, but says nothing about error behavior on a missing/invalid ID or how large the returned object is. With the output schema present, the remaining gap is modest, so a 3 is fair.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler, stating the action, the object domain, and the lookup key. Nothing could be trimmed without losing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Because an output schema exists, return values need not be described, and for a one-parameter read tool the definition is close to sufficient. The gaps are the missing usage/routing guidance and the vague resource name 'object' relative to a large sibling set of list/get tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for the single object_id parameter, so the description must compensate. Saying 'persistent ID' adds real meaning (durable, reusable reference) beyond the bare string type, but it does not explain the ID's format or where the agent obtains it, so it only partly fills the gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Retrieve') and resource, and usefully clarifies that an 'object' is a saved dataset or resolved plan, retrieved by persistent ID. It does not, however, differentiate itself from siblings such as objects_list, recipe_get, or dataset_inspect, leaving ambiguity about which getter to pick.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no mention of the alternative objects_list, and no note on prerequisites or what happens if the ID is unknown. The agent must infer the retrieval-by-ID pattern entirely from the schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
objects_listObjects ListCRead-only
Recover saved datasets and plans after reconnecting; returns bounded summaries.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | Yes | ||
| limit | No | ||
| offset | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already establishes that this is a safe read operation. The description adds that it returns bounded summaries, which is useful pagination context, but it does not explain limit/offset behavior or other operational details, so it adds only moderate value beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no wasted words. Its brevity is appropriate structurally, even though the content is under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with a required enum kind and optional pagination parameters, the description is too thin. An output schema exists so return values need not be described, but the input parameters and sibling alternatives are not adequately covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must carry parameter meaning, and it largely does not. 'Datasets and plans' loosely maps to some kind values, but it omits recipe_plan, experiment_plan, limit, and offset, leaving the enum and pagination parameters unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description says it recovers saved datasets and plans after reconnecting, which gives a vague sense of the resource and action. However, it uses 'recover' rather than 'list', and it does not mention the recipe_plan and experiment_plan kinds exposed by the schema, nor does it distinguish this tool from siblings like object_get or recipes_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'after reconnecting' hints at one usage scenario, but there is no explicit when-to-use versus alternatives guidance. It does not mention object_get for single-object retrieval, nor any conditions or exclusions for using this listing tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
recipe_getRecipe GetBRead-only
Inspect a recipe's typed configuration and execution requirements.
| Name | Required | Description | Default |
|---|---|---|---|
| recipe | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already establishes the safety profile, and an output schema exists, so return contents need not be described. The phrase 'typed configuration and execution requirements' adds some framing about what is inspected, but nothing about resolution failure behavior when a recipe name is unknown, which is the main behavioral gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with no filler, front-loaded on the action and resource. It is efficient, though it is arguably too terse given the undocumented parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema and readOnlyHint cover return shape and safety, so those need not be repeated. What remains missing is the identity/format of the 'recipe' argument and any hint about sibling selection, leaving the definition only minimally viable for invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single 'recipe' parameter carries no description in either the schema or the tool description. The description never clarifies whether this is a recipe name, ID, alias, or path, so it fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Inspect') and resource ('a recipe') plus what is returned: 'typed configuration and execution requirements'. However, it does not distinguish itself from close siblings like recipes_list, recipe_plan, or recipe_start, which an agent could easily confuse for this operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus recipes_list (enumerate recipes) or recipe_plan/recipe_start (act on a recipe). No prerequisites or conditions are stated, leaving routing entirely to the agent's inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
recipe_planRecipe PlanB
Validate and freeze an exact config for one allowlisted Cookbook recipe.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes | Exact Cookbook config values for one allowlisted recipe descriptor. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=false signals a mutating operation, and 'freeze' is consistent with that (no contradiction). However, the description never explains what 'freeze' actually does behaviorally — whether it persists a plan, returns a handle, or mutates remote state — nor does it note the allowlist constraint's failure mode. It adds some context but leaves the core behavior opaque.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with the action and scope front-loaded and no filler. It is appropriately sized, though it arguably omits useful routing context rather than being wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, and annotations cover the mutation hint. But for a nested-object tool with an idempotency_key and a planning/starting workflow, the description is too thin on when to call it relative to recipe_start to be fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single top-level 'request' object is fully documented in the schema, including the nested max_duration_seconds note. The description adds no parameter-level meaning beyond what the schema already provides, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States specific verbs (validate, freeze) and a specific resource (an exact config for an allowlisted Cookbook recipe). An agent can distinguish it from recipes_list/recipe_get, but it does not explicitly contrast with the closest sibling, recipe_start, so the boundary between planning and starting is left implicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use guidance, no mention of prerequisites (e.g. must the recipe appear in recipes_list first?), and no routing to alternatives like recipe_start. The pre-execution intent is only implied by the word 'validate'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
recipes_listRecipes ListCRead-only
Discover recipe requirements, availability and verification status.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already declares this is a safe read, so the description owes little on the safety front. Beyond that it adds no behavioral context — no mention of scope (all recipes vs. filtered), pagination, or whether verification status reflects live or cached checks. It essentially restates the resource surface.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with no filler or redundancy. It is appropriately sized, though it is a fragment built on a vague verb rather than a front-loaded statement of what is returned.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With zero parameters and an output schema present, the description needn't explain return values, and it gives a reasonable high-level summary of the surfaced fields. However, it omits any usage context, which matters given the overlapping recipe_get/recipe_plan siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the schema burden is nil and the baseline is 4. No parameter explanation is needed or expected.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the resource (recipes) and the attributes surfaced (requirements, availability, verification status), but the verb 'Discover' is vague for what is clearly a listing operation, and it does not distinguish itself from the sibling recipe_get. An agent can infer it returns a collection of recipes, but only because of the tool name, not the description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to use this tool, when to prefer recipe_get or recipe_plan instead, or any prerequisite context. With a similarly named sibling (recipe_get) in the toolset, the absence of routing guidance is a real gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
recipe_startRecipe StartAIdempotent
Submit a reviewed official Cookbook recipe. This may spend Tinker credits.
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | ||
| idempotency_key | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and idempotentHint=true, so the mutation and retry-safety profile is covered. The description adds genuinely new context by warning that the call 'may spend Tinker credits', a cost side effect not captured by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the action and followed by the cost warning. Nothing is wasted or buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained. But for a mutation tool that costs credits, the description leaves parameter meaning (plan_id, idempotency_key) entirely unexplained and gives no indication of what happens on failure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for two required parameters, and the description does not explain what plan_id references or how idempotency_key should be generated. With 2 undocumented params, the description fails to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (submit/start) and resource (an official Cookbook recipe), which distinguishes it from sibling reads like recipes_list and recipe_get. It does not explicitly differentiate itself from recipe_plan or training_start, but the action is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'reviewed official Cookbook recipe' implies a prerequisite state (the recipe must already be reviewed), which is useful implied guidance. However, it never names when to prefer this over recipe_plan or training_start, nor any exclusion conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sampleSampleCRead-only
Sample a base model or checkpoint through the model-recommended Cookbook renderer.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds only that the renderer defaults to the 'model-recommended Cookbook renderer', a small but real behavioral detail. It omits any mention of generation cost, latency, determinism/seed behavior, or how sampling and num_samples affect output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no wasted words. Efficient, though brevity here trades away necessary detail for a tool this complex.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a nested, multi-field sampling tool with 0% schema coverage, the description leaves most invocation-critical detail unstated. The output schema covers return values, but nothing tells the agent how to populate sampling, num_samples, or the target/renderer selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% on a deeply nested request object. The description hints at 'target' (model or checkpoint) and 'renderer', but says nothing about sampling controls (temperature, top_p, top_k, max_tokens, seed, stop), num_samples, or include_logprobs, leaving most parameters undocumented in both schema and description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Sample') and resource ('a base model or checkpoint'), which is clearer than the tautological title 'Sample'. However, it does not name or distinguish itself from the closest sibling, compute_logprobs, which also runs inference against a model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no when-to-use guidance or exclusions. An agent cannot tell from the text when to reach for 'sample' versus compute_logprobs, evaluate, or dataset_render_preview — all of which operate on models/checkpoints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
session_getSession GetARead-only
Inspect one owned Tinker session and its remote run identifiers.
| Name | Required | Description | Default |
|---|---|---|---|
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already tells the agent this is a safe read, so the description's main added value is the ownership scoping implied by 'owned' — a meaningful access constraint. It does not disclose error behavior, what happens on a non-owned or missing session_id, or any rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the resource and returned scope come first. It is perhaps terse to the point of omitting useful routing detail, but nothing in it is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read tool with readOnlyHint and an output schema covering return values, the description supplies the essential scope. The remaining gap is routing versus sessions_list and session_trace_export, which is minor given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the schema does not explain session_id at all; the description only implies it identifies a single owned session. It adds marginal meaning (ownership requirement) but no format, source, or validity guidance to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Inspect') and resource ('one owned Tinker session'), plus the returned payload type ('remote run identifiers'). 'One' distinguishes it from sessions_list implicitly, but the sibling is never named, so differentiation relies on the agent's inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The singular 'one ... session' implies a by-ID lookup rather than enumeration, which is enough to infer use over sessions_list. However, no explicit when-to-use condition, prerequisites, or alternatives are stated, and no guidance on when to prefer session_trace_export for deeper inspection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sessions_listSessions ListCRead-only
List sessions owned by the server's Tinker account.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| offset | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already establishes a safe read operation. The description adds the server-account ownership scope, which is useful context beyond annotations, but it does not disclose pagination behavior or response shape details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no wasted words. However, it is perhaps too terse given the undocumented parameters, though conciseness itself is strong.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema exists, so return values need not be explained, and readOnlyHint covers safety. But with 0% parameter description coverage and no usage guidance, an agent lacks context for filtering and pagination beyond the bare parameter names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate by explaining limit and offset. It provides no parameter meaning at all, leaving the two pagination parameters semantically undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (sessions) with an ownership scope constraint. It clearly contrasts with session_get by being a list operation, though it does not explicitly name sibling alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus session_get, session_trace_export, or other sibling listing tools. The ownership constraint is contextual but not usage-routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
session_trace_exportSession Trace ExportB
Save an owned session trace as a local artifact for bounded inspection.
| Name | Required | Description | Default |
|---|---|---|---|
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=false is consistent with the description's claim that it saves an artifact. The description adds useful context (output is a local artifact, scope is limited to owned sessions), but omits whether this persists to a specific location, overwrites existing files, or requires write permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no wasted words. Efficient, though arguably too terse to carry the required semantics for a tool with an undocumented parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation. As a one-parameter export tool the description is adequate at a minimum, but it leaves the parameter unexplained and gives no behavioral detail about where or how the artifact is saved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single parameter session_id is never mentioned in the description. The phrase 'owned session trace' loosely implies a session identifier is required, but the description does not compensate for the undocumented parameter, which is especially weak given its low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: saving a session trace as a local artifact. 'Owned' adds a scope qualifier. It is clear on its own but does not explicitly differentiate from sibling session tools like session_get or sessions_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'For bounded inspection' hints at the intended use case, implying you export when you want to examine a trace offline. However, it never states when to use this over session_get or other session tools, nor any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
train_distillTrain DistillC
Run on-policy or off-policy teacher/student distillation. Spends credits.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes | ||
| background | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description bears the full burden. It does disclose one critical behavior: 'Spends credits.' That's genuinely useful cost context, which pushes it above a 1. But for a training job with a heavy nested schema, checkpointing, background execution, and evaluation options, the description says nothing about runtime, whether it blocks, how to monitor, what gets created, or what permission/account state is required.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely tight: two short sentences, front-loaded with the operation, followed by the cost disclosure. Nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a high-complexity training tool with nested objects, checkpointing, evaluation, and background execution, the description is drastically incomplete. Output schema exists so return values need not be explained, but the agent still lacks any context on prerequisites, behavior, monitoring, or parameter meaning.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides zero parameter guidance. For a nested schema with dozens of fields across training/distillation/checkpointing blocks, this is a severe gap. The description doesn't even hint at the required 'request' object structure or the teacher/student fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (distillation training) and mentions on-policy/off-policy modes, which are the field's key axes. But it doesn't distinguish itself from siblings like train_sft, train_dpo, train_rl beyond being distillation specifically. The purpose is decipherable but requires the reader to already know what 'distillation' means in this context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use, when-not-to-use, or alternatives mentioned. The agent must infer that this is for teacher/student distillation from the name alone, and gets no guidance on when to prefer this over the sibling training tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
train_dpoTrain DpoC
Train on chosen/rejected pairs using Cookbook DPO. Spends credits.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes | ||
| background | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It does disclose that the operation spends credits, which is useful behavioral context, but omits other important traits such as expected duration, async/background behavior, idempotency, or resource requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core action and data requirement, with no wasted words. The credit warning is also concise and relevant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex DPO training tool with a large nested input schema and no annotations, the description is far too sparse. An output schema exists so return values need not be explained, but the agent gets almost no help constructing the required request object.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across a deeply nested request object with dozens of fields. The description only implies the dataset should contain chosen/rejected pairs and gives no guidance on model, dataset type, training hyperparameters, checkpointing, or background mode.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb (Train) and method (DPO) with the required data shape (chosen/rejected pairs), making the core operation clear. It does not explicitly differentiate itself from siblings like train_sft or train_rl, so it falls short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'Train on chosen/rejected pairs' implies the data precondition for using this tool, and 'Spends credits' gives a practical cost warning. However, it never states when to prefer this over train_sft, train_rl, or train_distill, nor any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_getTraining GetCRead-only
Get a Tuner workflow record or a Tinker training run by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | ||
| source | No | auto |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint=true annotation already tells the agent this is a safe, non-mutating read, so the description carries less burden. However, it adds nothing behavioral beyond that — no mention of what happens on a missing ID, what the two source systems imply, pagination, or permissions. The two-store framing is more purpose than behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the verb, both resource types, and the lookup key are all in the first clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need no prose, but with 0% parameter documentation the description leaves the source-selection behavior and ID semantics undefined for a tool spanning two distinct backends. Given the rich sibling set, it is too thin to route or invoke confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for both parameters, so the description must compensate and does not: run_id format (ID from which system?) and the source parameter (default 'auto', with no enumeration) are wholly unexplained. Critically, the meaning of run_id depends on the unstated source resolution, which is the key ambiguity an agent would need resolved.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Get) and resource (Tuner workflow record / Tinker training run) keyed by ID, which clearly distinguishes it from the sibling training_list and training_stop. It does not, however, differentiate itself from recipe_get or checkpoint_get, the other singular 'get by ID' tools in the family.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no indication of when to use this retrieval tool versus training_list, training_metrics, training_logs, or recipe_get, and states no precondition beyond the word 'by ID'. The agent must infer routing from sibling names alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_listTraining ListBRead-only
List remote Tinker runs or persistent local Tuner workflow records.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| offset | No | ||
| source | No | tinker |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds real context by disclosing that results come from two distinct sources (remote Tinker runs vs. persistent local Tuner records), but says nothing about result ordering, pagination behavior, or what happens when run history is unavailable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler. The core distinction is placed first and nothing is repeated from the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained. However, for a list tool with three entirely undocumented parameters, the description leaves the agent guessing about source values and paging, making it only minimally sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for all three parameters. The description hints at the 'source' axis (remote vs. local) but never maps it to the parameter or its values, and gives no meaning for limit or offset. With low coverage, the description needed to compensate more than it does.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb ('List') and a specific resource ('remote Tinker runs or persistent local Tuner workflow records'), which separates it from checkpoint_list, sessions_list, and recipes_list. The only weakness is that it does not explicitly name the sibling list tools it differs from, relying on jargon ('Tinker', 'Tuner') that an agent may not resolve on its own.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to call this versus training_get, training_metrics, training_logs, or sessions_list, nor any note on prerequisites or pagination workflow. The reader must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_logsTraining LogsCRead-onlyIdempotent
Read bounded recursive logs, or an artifact path with a byte cursor.
| Name | Required | Description | Default |
|---|---|---|---|
| cursor | No | ||
| offset | No | ||
| run_id | Yes | ||
| max_lines | No | ||
| artifact_path | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, lowering the bar. The description adds genuine context beyond them by disclosing two access modes (bounded recursive log reads vs. artifact-path reads with a byte cursor), but says nothing about auth, limits, or what the cursor returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single compact, front-loaded sentence with no filler. It is appropriately sized, though its brevity borders on cryptic rather than crisp.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, but with 5 parameters at 0% schema coverage and no annotation-derived param detail, the description leaves the calling contract largely undefined for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 5 parameters, so the description must carry the load. It only loosely gestures at artifact_path and cursor; run_id, offset, and max_lines (the bound implied by "bounded") are never explained in either place.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The verb+resource ("Read ... logs") is identifiable, but the object is garbled: "bounded recursive logs, or an artifact path with a byte cursor" reads as two half-merged modes. It gives no differentiation from siblings like training_metrics, training_get, or session_trace_export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use, when-not-to-use, or named alternative among the ~48 siblings. The phrase "or an artifact path" hints at a second mode but never states when to pick it over the default recursive-log mode.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_metricsTraining MetricsARead-onlyIdempotent
Read latest metrics, or page from a byte cursor (start at zero) while training runs.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| cursor | No | ||
| run_id | Yes | ||
| artifact_path | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so safety is covered. The description adds genuinely useful behavior beyond that: cursor-based paging with a byte cursor and the 'start at zero' convention, plus the fact that it can be polled during a live run. It stops short of describing pagination termination or refresh/consistency behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single efficient sentence that front-loads the primary read action and appends the paging qualifier. Every clause earns its place, with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values need not be documented, and annotations cover the safety profile. The gap is parameter coverage: an opaque parameter like artifact_path is left undefined, and the relationship between cursor and limit paging is only half-specified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across 4 parameters, so the description must carry the burden. It adds real meaning for 'cursor' (byte cursor, start at zero) but leaves 'run_id', 'limit', and especially 'artifact_path' entirely unexplained in both schema and description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb and resource ('Read latest metrics') plus the paging capability and the context in which it applies ('while training runs'). It does not differentiate itself from the nearby sibling training_logs, which an agent could easily confuse with a metrics-reading tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'while training runs' implies the intended context for use, and the mention of paging implies the long-poll/streaming case. However, no alternatives are named (e.g., training_logs or training_get) and no when-not-to-use condition is given, so guidance remains implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_planTraining PlanA
Prepare an immutable training plan and report blockers; does not launch training.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes | ||
| objective | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=false, so the description carries the burden and does well: it tells the agent the plan is immutable (no post-hoc edits) and that it reports blockers rather than silently succeeding. It omits what happens on repeat calls (idempotency_key exists in schema) and any permission needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence, front-loaded with the core action and ending with the boundary condition. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need no explanation, and the description correctly conveys that planning is non-destructive. But for a tool whose single input is a complex discriminated union of four method shapes, the description never explains the method variants or how 'objective' interacts, leaving it thin relative to the schema's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description says nothing about the two parameters. The large nested 'request' union (sft/dpo/rl/distill variants, training hyperparameters, idempotency_key) and the 'objective' string are entirely undocumented in prose, so the description does not compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Prepare an immutable training plan') and adds a negative scope statement ('does not launch training') that separates it from training_start/train_sft neighbors. It does not name an alternative tool explicitly, so it falls just short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'does not launch training' implicitly tells the agent this is the pre-launch planning step, and 'report blockers' hints at a validation use case. However, it never says when to choose this over recipe_plan, experiment_autoplan, or training_start, so usage is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_resumeTraining ResumeBIdempotent
Resume SFT with total max_steps or additional_steps after checkpoint. Spends credits.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | ||
| max_steps | No | ||
| num_epochs | No | ||
| idempotency_key | Yes | ||
| additional_steps | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false (a mutation) and idempotentHint=true, so the safety profile is partly covered. The description adds two useful facts beyond annotations: it spends credits and requires a prior checkpoint. It omits what happens to prior progress, permission needs, or failure behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences with no waste. The core action and its parameter choice are front-loaded, and the cost warning follows. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and the checkpoint/credit notes are valuable. Still, as a credit-spending mutation with 0% schema coverage, the description leaves num_epochs and idempotency_key semantics and the interaction between max_steps and num_epochs unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% for 5 parameters. The description names max_steps ('total') and additional_steps (incremental) and clarifies their semantic difference, which is genuinely helpful. But run_id, num_epochs, and idempotency_key get no explanation, leaving most parameters undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Resume SFT') and distinguishes itself from the training_start/train_sft siblings by implying continuation rather than initiation. The acronym 'SFT' and lack of explicit sibling naming keep it from a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'after checkpoint' implies the prerequisite state (a checkpoint must exist) and it surfaces the max_steps-vs-additional_steps choice. However, it never states when to use this over training_start or training_stop, nor any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_startTraining StartAIdempotent
Submit a validated plan to Docket and return its run ID promptly. Spends credits.
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | ||
| idempotency_key | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations cover the safety profile (readOnlyHint=false) and idempotency (idempotentHint=true), so the description adds genuinely new context: the operation 'Spends credits' and returns a run ID 'promptly,' signaling async, billable execution. This cost/async disclosure is valuable beyond the structured fields, though credit magnitude and failure behavior are unstated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with no filler; the core action is front-loaded and the cost warning is appended efficiently. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values needn't be explained, and annotations already declare the mutation/idempotency profile. Still, for a billable mutation tool the description leaves parameter meaning and what counts as a 'validated plan' undefined, so it is only minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the full burden for the two parameters. It never explains plan_id or idempotency_key, leaving the idempotency contract to be inferred from the annotation alone. With both params undocumented, the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Submit a validated plan to Docket and return its run ID.' An agent can tell it launches a training run from a plan. It does not explicitly distinguish itself from siblings like recipe_start or training_resume, but the 'validated plan' scoping is fairly clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies the prerequisite that the plan must already be validated, which nudges the agent toward training_plan first. However, it names no alternatives and gives no explicit when/when-not guidance versus training_resume, recipe_start, or train_sft.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
training_stopTraining StopBIdempotent
Stop local orchestration. Already submitted remote work may continue.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate a write operation (readOnlyHint=false) and idempotency (idempotentHint=true). The description adds a meaningful behavioral nuance: stopping local orchestration does not necessarily halt already submitted remote work, which is important for understanding partial effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded sentences with no waste. The action is stated first, then a clarifying constraint, and every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and annotations cover safety and idempotency. However, the description omits any mention of the required run_id and provides no usage context, leaving clear gaps for an agent selecting and invoking the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, but it says nothing about the required run_id parameter. The parameter name is somewhat self-explanatory in context, but the description adds no explicit meaning or format guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (stop) and resource (local orchestration), and the second sentence clarifies scope. It does not explicitly name 'training' or 'run', but the tool name and title supply that context. It distinguishes from training_start and training_resume, though the resource phrase is somewhat abstract.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no explicit when-to-use guidance or alternatives. The second sentence only states a behavioral caveat, not when this tool should be chosen over siblings like training_resume or training_start.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
train_rlTrain RlC
Run an allowlisted arithmetic/math group-rollout RL recipe. Spends credits.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes | ||
| background | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It usefully notes that the operation 'Spends credits,' but says nothing about long-running behavior, background execution, permissions, side effects, or failure modes expected of a training job.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two front-loaded sentences with no wasted words: purpose first, cost warning second. It is efficiently structured, though its extreme brevity for such a complex tool borders on under-specification.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described. However, with no annotations, 0% schema coverage, nested objects, and many parameters, the description is far too thin to guide correct invocation; it omits usage conditions, parameter semantics, and most behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the tool has a deeply nested request object with many configurable fields. The description adds almost no parameter meaning beyond the words 'arithmetic/math' and 'group-rollout,' leaving the agent to infer the roles of model, recipe, training, evaluation, checkpointing, and background from property names alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Run') and resource ('allowlisted arithmetic/math group-rollout RL recipe'), clearly identifying an RL training operation. It distinguishes itself from sibling training tools like train_sft and train_dpo by specifying RL, but it does not explicitly name alternatives or contrast with them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives such as training_start, recipe_start, train_sft, train_dpo, or train_distill. The phrase 'allowlisted arithmetic/math' hints at a constraint but does not tell the agent which scenarios call for this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
train_sftTrain SftC
Run Cookbook supervised fine-tuning. This operation spends credits.
| Name | Required | Description | Default |
|---|---|---|---|
| request | Yes | ||
| background | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral burden. It usefully discloses that the operation spends credits, a key side effect, but omits other important traits such as long-running execution, background processing, output format, idempotency, and checkpointing implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with zero waste. Purpose is front-loaded, followed by the critical cost warning. Nothing extraneous.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with a deeply nested schema, no annotations, and multiple sibling training methods, the description is far too sparse. It gives purpose and cost but no guidance on required inputs, background behavior, or how the operation relates to alternatives.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% across a large nested request object, and the description adds no parameter meaning. It does not explain the required model/dataset structure, background flag, or any nested training/evaluation/checkpointing fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Run Cookbook supervised fine-tuning.' The 'SFT' method distinguishes it from sibling training tools like train_dpo, train_rl, and train_distill, though no sibling is named explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a cost warning ('spends credits') but gives no guidance on when to choose SFT over DPO, RL, or distillation, nor any prerequisite or context for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
usage_getUsage GetARead-only
Get account usage for a half-open YYYY-MM-DD date range; preserve upstream units.
| Name | Required | Description | Default |
|---|---|---|---|
| starting_on | Yes | ||
| ending_before | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds the half-open range semantics and 'preserve upstream units,' which is useful behavioral context beyond the schema. However, it does not disclose what the response contains or whether any aggregation/formatting occurs, so it adds only modest value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the core action and resource, then adds the key scoping detail. Every word earns its place; there is no fluff or repetition of schema field names.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with two parameters and an output schema, the description covers the essential semantics: what is fetched and the date-range behavior. The output schema presumably documents the return shape, so the lack of return details is acceptable. It could be more complete with an explicit 'use this for account-level usage' note, but it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does clarify that the two parameters form a half-open date range, which is meaningful. However, it does not explain the format constraints beyond YYYY-MM-DD or the relationship between starting_on and ending_before in detail, so the compensation is partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Get') and resource ('account usage'), and adds a precise scope: a half-open YYYY-MM-DD date range. It does not explicitly distinguish itself from sibling tools, but the resource is clear enough that an agent can infer it is a read-only usage query.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by naming the date-range parameters, but it does not state when to use this tool versus alternatives or mention any exclusions. The half-open range hint is useful, but there is no explicit guidance about prerequisites or when a different tool would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
3 tool updates
v0.1.1- Changed
checkpoint_export1 field changed- added
Input schema / properties / backgroundAdded value: +{ + "default": true, + "type": "boolean" +}
- Changed
dataset_fetch_hf6 fields changed- added
Input schema / properties / request / properties / chosen_fieldAdded value: +{ + "default": "chosen", + "minLength": 1, + "type": "string" +} - added
Input schema / properties / request / properties / invalid_record_policyAdded value: +{ + "default": "error", + "description": "Fail on invalid rows, or skip them and report removal counts", + "enum": [ + "error", + "skip" + ], + "type": "string" +} - added
Input schema / properties / request / properties / preference_prompt_fieldAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Row field for the shared preference prompt, e.g. 'instruction'" +} - added
Input schema / properties / request / properties / rejected_fieldAdded value: +{ + "default": "rejected", + "minLength": 1, + "type": "string" +} - added
Input schema / properties / request / properties / tool_calls_fieldAdded value: +{ + "anyOf": [ + { + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Row field containing target function calls, including raw xLAM answers" +} - added
Input schema / properties / request / properties / tools_fieldAdded value: +{ + "anyOf": [ + { + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Row field containing OpenAI or raw function declarations, including JSON text" +}
- Changed
dataset_prepare7 fields changed- added
Input schema / properties / request / properties / chosen_fieldAdded value: +{ + "default": "chosen", + "minLength": 1, + "type": "string" +} - added
Input schema / properties / request / properties / inline_recordsAdded value: +{ + "anyOf": [ + { + "items": { + "additionalProperties": {}, + "type": "object" + }, + "maxItems": 1000, + "minItems": 1, + "type": "array" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Inline synthetic or user-authored records to validate and persist without a server-local file" +} - added
Input schema / properties / request / properties / invalid_record_policyAdded value: +{ + "default": "error", + "description": "Fail on invalid rows, or skip them and report removal counts", + "enum": [ + "error", + "skip" + ], + "type": "string" +} - added
Input schema / properties / request / properties / preference_prompt_fieldAdded value: +{ + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Row field for the shared preference prompt, e.g. 'instruction'" +} - added
Input schema / properties / request / properties / rejected_fieldAdded value: +{ + "default": "rejected", + "minLength": 1, + "type": "string" +} - added
Input schema / properties / request / properties / tool_calls_fieldAdded value: +{ + "anyOf": [ + { + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Row field containing target function calls, including raw xLAM answers" +} - added
Input schema / properties / request / properties / tools_fieldAdded value: +{ + "anyOf": [ + { + "minLength": 1, + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "description": "Row field containing OpenAI or raw function declarations, including JSON text" +}
48 tool updates
v0.1.0- First observed
benchmarks_list - First observed
capabilities_get - First observed
checkpoint_delete - First observed
checkpoint_export - First observed
checkpoint_get - First observed
checkpoint_list - First observed
checkpoint_publish - First observed
checkpoint_set_ttl - First observed
checkpoint_unpublish - First observed
compare_runs - First observed
compute_logprobs - First observed
dataset_fetch_hf - First observed
dataset_inspect - First observed
dataset_prepare - First observed
dataset_probe_hf - First observed
dataset_render_preview - First observed
dataset_search_hf - First observed
dataset_validate - First observed
evaluate - First observed
evaluation_failures - First observed
evaluation_get - First observed
experiment_artifacts - First observed
experiment_autoplan - First observed
experiment_rollouts - First observed
models_list - First observed
object_get - First observed
objects_list - First observed
recipe_get - First observed
recipe_plan - First observed
recipe_start - First observed
recipes_list - First observed
sample - First observed
session_get - First observed
session_trace_export - First observed
sessions_list - First observed
train_distill - First observed
train_dpo - First observed
train_rl - First observed
train_sft - First observed
training_get - First observed
training_list - First observed
training_logs - First observed
training_metrics - First observed
training_plan - First observed
training_resume - First observed
training_start - First observed
training_stop - First observed
usage_get
TDQS
Scored across 48 tools
Most tools are grouped by resource and action, but the large set has several close planning/staging tools (experiment_autoplan/training_plan/recipe_plan, dataset_fetch_hf/dataset_probe_hf/dataset_prepare, dataset_validate/dataset_inspect) that could cause misselection. Descriptions help, but boundaries are not always obvious.
Names consistently use snake_case and resource prefixes, but action position varies: list/get tools are noun_verb (recipes_list, checkpoint_get) while training actions are verb_noun (train_sft, compute_logprobs) and some are bare verbs (evaluate, sample). The prefix grouping keeps it readable, but the mixed conventions are not fully predictable.
48 tools is far beyond the 25+ threshold and would overwhelm an agent even though the domain spans training, datasets, checkpoints, and evals. Several narrow operations (dataset_validate vs dataset_inspect, checkpoint_* variants) could be consolidated.
The surface covers the full training lifecycle: datasets, recipes, plan/start/resume/stop training, multiple RL/DPO/SFT/distill methods, checkpoint management, evaluation, sessions, and usage. Minor gaps exist (no evaluation_list, no training/cancel/delete for remote runs, no dataset deletion), but agents can complete core workflows.
Maintenance
Related MCP Connectors
Discover, preview, estimate, run, and retrieve reusable AI workflows.
Build, validate, and deploy multi-agent AI solutions from any AI environment.
Connect, monitor, and control AI agents — tasks, approvals, schedules, and governance.
The cloud for agents. Tools for AI agents to register, build, and deploy other agents. Zero human required.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceTransforms AI assistants into a full ML engineering environment for training and fine-tuning models across multiple backends (local GPU, Mistral, Together AI, OpenAI) and cloud providers (Lambda Labs, RunPod, SSH-accessible VPS), with dataset management, experiment tracking, cost estimation, and deployment to Ollama/Open WebUI.3PolyForm Noncommercial 1.0.0
- AlicenseNot gradedqualityDmaintenanceProvides access to over 500 pre-configured YAML templates and guided workflows for fine-tuning, training, and evaluating LLMs like Llama and DeepSeek. It enables AI assistants to search for recipes, retrieve configurations, and validate parameters for various machine learning tasks.Apache 2.0
- FlicenseNot gradedqualityBmaintenanceEnables LLMs to manage and run machine learning training jobs on a remote server, including syncing code, submitting experiments, monitoring progress, reading TensorBoard metrics, and receiving completion notifications.-
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to manage GPU training end-to-end through natural language, including submitting and scheduling jobs, monitoring logs and metrics, diagnosing failures, comparing runs, and recommending the best checkpoints.Apache 2.0