Skip to main content
Glama

Tuning Engines CLI & MCP Server

tuning-engines-cli MCP server

npm version MCP Registry License: MIT

Govern every AI workflow through one API.

Tuning Engines is a governed AI runtime for model, agent, skill, and MCP workflows. Route inference through one OpenAI-compatible API, apply RBAC and traffic policies, request approvals for high-risk actions, inspect traces and usage, and connect durable orchestration frameworks such as LangGraph and Temporal. The same CLI and MCP server also manage domain-specific fine-tuning of open-source models.

Training Agents

Tuning Engines uses specialized agents that control how your data is analyzed and converted into training data. Each agent produces a different kind of domain-specific fine-tuned model optimized for its use case. Current agents focus on code, with more coming for customer support, data extraction, security review, ops, and other domains.

Cody (code_repo) — Code Autocomplete Agent

Cody fine-tunes on your GitHub repo using QLoRA (4-bit quantized LoRA) via the Axolotl framework (HuggingFace Transformers + PEFT). It learns your codebase's patterns, naming conventions, and project structure to produce a fast, lightweight adapter optimized for real-time completions.

Best for: code autocomplete, inline suggestions, tab-complete, code style matching, pattern completion.

te jobs create --agent code_repo \
  --base-model Qwen/Qwen2.5-Coder-7B-Instruct \
  --repo-url https://github.com/your-org/your-repo \
  --output-name my-cody-model

SIERA (sera_code_repo) — Bug-Fix Specialist

SIERA (Synthetic Intelligent Error Resolution Agent) uses the Open Coding Agents approach from AllenAI to generate targeted bug-fix training data from your repository. It synthesizes realistic error scenarios and their resolutions, then fine-tunes a model that learns your team's debugging style, error handling conventions, and fix patterns.

Best for: debugging, error resolution, patch generation, root cause analysis, fix suggestions.

te jobs create --agent sera_code_repo \
  --quality-tier high \
  --base-model Qwen/Qwen2.5-Coder-7B-Instruct \
  --repo-url https://github.com/your-org/your-repo \
  --output-name my-siera-model

Quality tiers (SIERA only):

  • low — Faster, fewer synthetic pairs (default)

  • high — Deeper analysis, more training data, better results

Coming Soon

Agent

Persona

What it does

Resolve

Mira

Fine-tunes on support tickets, macros, and KB articles for automated ticket resolution

Extractor

Flux

Trains for strict schema extraction from docs, PDFs, and business text

Guard

Aegis

Security-focused code reviewer that catches risky patterns and proposes safer fixes

OpsPilot

Atlas

Incident response agent trained on runbooks, postmortems, and on-call notes

Related MCP server: ML Lab MCP

Supported Base Models

Size

Models

3B

Qwen/Qwen2.5-Coder-3B-Instruct

7B

codellama/CodeLlama-7b-hf, deepseek-ai/deepseek-coder-7b-instruct-v1.5, Qwen/Qwen2.5-Coder-7B-Instruct

13-15B

codellama/CodeLlama-13b-Instruct-hf, bigcode/starcoder2-15b, Qwen/Qwen2.5-Coder-14B-Instruct

32-34B

deepseek-ai/deepseek-coder-33b-instruct, codellama/CodeLlama-34b-Instruct-hf, Qwen/Qwen2.5-Coder-32B-Instruct

70-72B

codellama/CodeLlama-70b-Instruct-hf, meta-llama/Llama-3.1-70B-Instruct, Qwen/Qwen2.5-72B-Instruct

Quick Start

npm install -g tuningengines-cli

# Or run without installing
npx -y --package tuningengines-cli@latest te auth status

# Sign up or log in (opens browser — works for new accounts too)
te auth login

# Add credits (opens browser to billing page)
te billing add-credits

# Estimate cost before training
te jobs estimate --base-model Qwen/Qwen2.5-Coder-7B-Instruct

# Train Cody on your repo
te jobs create --agent code_repo \
  --base-model Qwen/Qwen2.5-Coder-7B-Instruct \
  --repo-url https://github.com/your-org/your-repo \
  --output-name my-model

# Monitor training
te jobs status <job-id> --watch

# View your trained models
te models list

# Create a governed orchestration starter
te orchestration init langgraph
te orchestration init temporal
te orchestration init inngest
te orchestration init triggerdev
te orchestration init hatchet
te orchestration init restate
te orchestration init dbos
te orchestration init dapr
te orchestration init prefect
te orchestration init dagster
te orchestration init airflow

MCP Server Setup

The CLI includes a built-in MCP server with 60+ tools. Any AI assistant that supports MCP can fine-tune models, manage training jobs, run evaluations, check inference usage, inspect traces, review approvals, and manage non-secret tenant registry metadata through natural language.

For security, the MCP server intentionally does not expose internal proxy routes. It also refuses MCP-side inference-key creation and raw secret-bearing mutation fields. Use the CLI or web UI for workflows that intentionally create one-time keys, submit raw provider secrets, validate S3 credentials, or import/export S3 assets with raw credentials.

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "tuning-engines": {
      "command": "npx",
      "args": ["-y", "--package", "tuningengines-cli@latest", "te", "mcp", "serve"],
      "env": {
        "TE_API_KEY": "te_your_key_here"
      }
    }
  }
}

Claude Code

claude mcp add tuning-engines -- npx -y --package tuningengines-cli@latest te mcp serve

Work Sessions and outcomes

Label the desired outcome for a project without interrupting your coding workflow:

te goal start "Fix flaky checkout retries"
te goal show
te goal complete --result succeeded

Install optional native telemetry hooks for Claude Code or Codex:

te guard claude-code install --mode observe --project .
te guard claude-code doctor
te guard claude-code doctor --probe
te guard codex install
te guard codex doctor

Claude Code writes project-local hooks into .claude/settings.local.json. On Windows, verify with dir .\.claude, type .\.claude\settings.local.json, then restart Claude Code from the same project root and review claude /hooks. doctor --probe is available in tuningengines-cli 0.4.20 and later; it runs synthetic hook events through the installed commands and checks that the trace is visible to Tuning Engines. Hook invocations also write a local redacted status log at .claude/tuning-engines-hook-status.jsonl. Codex project hooks require review and trust from /hooks. The installer pins hooks to the exact CLI executable that created them; te guard codex doctor reports that path, CLI version, native-event contract, required lifecycle hooks, and any conflicting te command found on PATH. Each Codex or Claude Code user prompt starts a new trace while retaining one stable Work Session for the native conversation. Tuning Engines sends pseudonymous session and transcript references by default, not transcript contents or local absolute paths.

Claude Code Plugin

The repository also ships a Claude Code plugin wrapper around the same MCP server. It keeps installation discoverable while preserving the same TE_API_KEY environment-variable boundary:

claude plugin marketplace add cerebrixos-org/tuning-engines-cli
claude plugin install tuning-engines@tuning-engines

The same plugin directory includes a Codex plugin manifest, marketplace entry, MCP server, and tenant-operations skill. It uses the Tuning Engines web app as the shared control-plane UI for traces, approvals, policies, cost, and review.

DeepSeek Harness Plugin

Install the native Harness bundle to capture turn, model-step, and tool lifecycle telemetry and optionally enforce TE governance before tool execution:

export TE_API_KEY="your-tenant-token-or-inference-key"
dsh plugin --profile default add tuningengines-cli

The adapter uses a disk-backed local spool, background batching, bounded retries, stable tool-call correlation, and metadata-only capture by default.

VS Code / Cursor / Windsurf

Add to your MCP settings (.vscode/mcp.json or equivalent):

{
  "servers": {
    "tuning-engines": {
      "command": "npx",
      "args": ["-y", "--package", "tuningengines-cli@latest", "te", "mcp", "serve"],
      "env": {
        "TE_API_KEY": "te_your_key_here"
      }
    }
  }
}

What the AI assistant can do

When connected, your AI assistant can:

  • "Fine-tune Qwen 7B on my-org/my-repo using the SIERA agent with high quality"

  • "How much would it cost to train a 32B model for 3 epochs on this repo?"

  • "Check the status of my latest training job"

  • "List my trained models"

  • "Export my model to s3://my-bucket/models/"

  • "Show my account balance"

  • "Train a bug-fix specialist on this repo" (auto-selects SIERA)

  • "Create an autocomplete model for this codebase" (auto-selects Cody)

The create_job tool description includes full agent details and model lists, so AI assistants automatically select the right agent and model based on what you ask for.

Unified API Endpoint

Tuning Engines can be used anywhere a tool accepts an OpenAI-compatible API base URL. Point the client at:

https://api.tuningengines.com/v1

Use an inference key that starts with sk-te-... for live model calls, and use the model IDs shown by:

te inference models

This lets OpenCode, Temporal activities, LangGraph apps, OpenAI SDK clients, and other custom-provider clients route through the same Tuning Engines control plane for model RBAC, routing, fallbacks, guardrails, AGT policy, traces, usage metering, and cost attribution.

See docs/unified-api-endpoint.md for copy-paste examples for OpenCode, Temporal, Python, JavaScript, and other OpenAI-compatible clients.

Agent Runtime SDK and Orchestration Starters

Use the CLI/MCP package when you want npx tools for assistants. Use the Python SDK when you want your own app to run durable agent workflows while Tuning Engines remains the governed control plane for models, agents, skills, MCP tools, RBAC, AGT policy, audit, usage, and token economics.

OpenAI Agents SDK users can install the native model and trace integration:

pip install "tuning-engines[openai-agents]"

See packages/tuning-agents/README.md for setup. The OpenAI Agents runtime owns the loop; TE supplies the governed endpoint and receives metadata-only SDK traces through its background processor.

Install the published Python SDK:

pip install "tuning-engines[langgraph]"
pip install "tuning-engines[temporal]"

The package installs the tuning_agents Python module.

LangGraph example:

from langgraph.checkpoint.memory import InMemorySaver

from tuning_agents import TuningClient
from tuning_agents.langgraph import create_tuning_langgraph_agent, invoke_with_trace

client = TuningClient(api_key="te_your_key_here")

agent = create_tuning_langgraph_agent(
    client,
    model="llama-3.3-70b-fp8",
    agent_names=["billing-escalation"],
    checkpointer=InMemorySaver(),
    interrupt_before=["tools"],
)

result = invoke_with_trace(
    client,
    agent,
    [{"role": "user", "content": "Triage this ticket and escalate if needed."}],
    thread_id="ticket-123",
)

client.flush_trace(name="ticket-triage", runtime="langgraph", status="succeeded")

Temporal example:

from tuning_agents.temporal import (
    TuningEnginesTemporalFeatures,
    create_tuning_engines_plugin,
    define_temporal_workflow,
)

plugin = create_tuning_engines_plugin(
    features=TuningEnginesTemporalFeatures(
        built_in_workflow=False,
        model_calls=True,
        skill_tools=True,
        mcp_tools=True,
        agents=True,
        approvals=True,
        traces=True,
        state_references=True,
        interventions=True,
        model_catalog=True,
        usage=True,
    )
)
TuningAgentWorkflow = define_temporal_workflow()
# Pass plugin to Client.connect(..., plugins=[plugin]) and register the workflow.

The SDK captures runtime events from LangGraph/Temporal and posts them to POST /api/v1/traces. Each event carries a run_id, request_id, and a normalized event type such as model.call, mcp.tool_call, agent.message, workflow.step, human.edit, action.finalized, outcome.recorded, or state.reference. The app pairs that with inference usage, request capture, policy decisions, approval requests, external state references, audit, and billing logs.

JavaScript/TypeScript users can also import lightweight tracing helpers from the npm package:

import { createOpenAIAgentsTraceAdapter } from "tuningengines-cli/adapters/openai-agents";
import { createClaudeAgentSdkTraceAdapter } from "tuningengines-cli/adapters/claude-agent-sdk";

Both helpers send redacted run, model, tool, handoff, error, goal, and outcome events to the existing trace API. goal_key, goal_status, and goal_score are normalized into the same success-signal analytics as outcome_key.

For decision traces, store redacted signals in metadata.decision, for example proposal_summary, changed_fields, change_summary, final_action, outcome_label, and reason_summary. Do not place raw prompts, provider keys, tenant secrets, or full customer data in trace metadata.

Generate a starter kit:

te orchestration init langgraph --dir ./lg-te-demo
te orchestration init temporal --dir ./temporal-te-demo
te orchestration init inngest --dir ./inngest-te-demo
te orchestration init triggerdev --dir ./trigger-te-demo
te orchestration init hatchet --dir ./hatchet-te-demo
te orchestration init restate --dir ./restate-te-demo
te orchestration init dbos --dir ./dbos-te-demo
te orchestration init dapr --dir ./dapr-te-demo
te orchestration init prefect --dir ./prefect-te-demo
te orchestration init dagster --dir ./dagster-te-demo
te orchestration init airflow --dir ./airflow-te-demo

LangGraph and Temporal starters use the Python runtime SDK. Inngest, Trigger.dev, and Hatchet starters generate TypeScript projects with a small self-contained Tuning Engines helper. Restate, DBOS, and Dapr starters use the same TypeScript helper. Prefect, Dagster, and Airflow starters generate Python workflow examples with a small helper module. All generated examples include governed model calls, trace flushing, registry manifests, policy context metadata, decision metadata, runtime state references, and approval retry patterns.

CLI Commands

Authentication

Command

Description

te auth login

Sign up or log in via browser

te auth logout

Clear saved credentials

te auth status

Show current auth status (email, balance)

Training Jobs

Command

Description

te jobs list

List all training jobs

te jobs show <id>

Show job details

te jobs create

Submit a training job (--agent, --quality-tier, --base-model, --repo-url, --output-name)

te jobs status <id>

Live status (--watch for continuous polling)

te jobs cancel <id>

Cancel a running job

te jobs retry <id>

Retry from last checkpoint

te jobs estimate

Cost estimate before submitting

te jobs validate-s3

Pre-validate S3 credentials

Models

Command

Description

te models list

List your trained models

te models show <id>

Show model details

te models base

List supported base models

te models import

Import a model from S3

te models export <id>

Export a model to S3

te models delete <id>

Delete a model

te models status <id>

Check import/export status

Datasets

Command

Description

te datasets list

List all datasets

te datasets show <id>

Show dataset details

te datasets create

Create a dataset from S3 (--name, --s3-url, --for-evaluation)

te datasets delete <id>

Delete a dataset

te datasets status <id>

Check import/processing status

Evaluations

Command

Description

te evals list

List all evaluations

te evals show <id>

Show evaluation details and scores

te evals create

Run an evaluation (--model, --dataset, --evaluators)

te evals cancel <id>

Cancel a running evaluation

te evals status <id>

Live evaluation progress

te evals evaluators

List available evaluators

te evals estimate

Cost estimate for an evaluation

Inference

Command

Description

te inference models

List available inference models

te inference usage

Show inference API usage stats

te inference jwt

Get a JWT for direct API access

te inference token

Exchange an inference key (sk-te-...) for a short-lived inference JWT

te inference chat --data '<json>'

Run an OpenAI-compatible chat completion

te inference responses --data '<json>'

Run an OpenAI Responses request

te inference embeddings --data '<json>'

Create embeddings

te inference messages --data '<json>'

Run an Anthropic-compatible Messages request

Runtime Traces and Approvals

Command

Description

te traces list

List LangGraph, Temporal, and custom runtime traces

te traces show <run-id>

Show one trace, including events, policy decisions, and approvals when linked

te traces ingest --data '<json>'

Ingest or update a trace using a user API token or inference key

te outcomes list

List observed outcomes, goals, evals, and workflow success signals

te outcomes record --run-id ... --key ... --label ...

Record a success signal for a run

te outcomes map --outcome-key ... --criteria '<json>'

Map unmapped events to an outcome key

te insights list

List Insight Loop recommendations

te insights accept <id>

Accept an insight as valid; does not change production

te insights apply <id>

Apply or queue the approved action for an accepted insight

te doctor simulate --data '<json>'

Simulate inference access, role, endpoint, policy, and resource checks

te policy-decisions list

List AGT YAML policy decisions

te policy-decisions show <id>

Show one policy decision with redacted context

te policy-templates list

List curated AGT YAML policy templates

te policy-templates render <id> --params '<json>'

Render disabled/shadow policy YAML from safe structured parameters

te policy-drafts generate --prompt '<text>'

Generate an AI-assisted disabled/shadow draft for review and testing

te approvals list --status pending

List policy approval requests

te approvals show <id>

Show approval detail and retry metadata

te approvals approve <id>

Approve a pending request

te approvals deny <id>

Deny a pending request

te interventions list/show/request/ack/complete/fail

Manage runtime pause, resume, cancel, and replay requests

te state list/show/upsert

Manage safe external workflow-state and memory references

te registry sync --file tuning-registry.yml --dry-run/--apply

Diff or apply agent, skill, and MCP registry manifests

te registry show <id>

Inspect an applied registry sync

te work-sessions list/show/complete/confirm-outcome

Inspect and update Work Sessions

te initiatives list/show/create/update

Manage strategic initiative groupings

Orchestration Starters

Command

Description

te orchestration init langgraph

Create a LangGraph starter wired to Tuning Engines governance and traces

te orchestration init temporal

Create a Temporal worker starter wired to Tuning Engines governance and traces

te orchestration init inngest

Create an Inngest function starter wired to Tuning Engines governance and traces

te orchestration init triggerdev

Create a Trigger.dev task starter wired to Tuning Engines governance and traces

te orchestration init hatchet

Create a Hatchet workflow starter wired to Tuning Engines governance and traces

te orchestration init restate

Create a Restate service starter wired to Tuning Engines governance and traces

te orchestration init dbos

Create a DBOS workflow starter wired to Tuning Engines governance and traces

te orchestration init dapr

Create a Dapr Workflow starter wired to Tuning Engines governance and traces

te orchestration init prefect

Create a Prefect flow starter wired to Tuning Engines governance and traces

te orchestration init dagster

Create a Dagster asset starter wired to Tuning Engines governance and traces

te orchestration init airflow

Create an Airflow DAG starter wired to Tuning Engines governance and traces

Agents

Command

Description

te agents list

List available agents

te agents show <id>

Show agent details and capabilities

te agents message <name> --data '<json>'

Send a governed A2A agent message

Skills and MCP Execution

Command

Description

te skills list

List skills visible to the inference identity

te skills prepare <name> --data '<json>'

Prepare a governed skill

te skills invoke <name> --data '<json>'

Invoke a governed skill

te mcp call --server <name> --tool <name> --arguments '<json>'

Call an enabled governed MCP tool

te mcp rediscover <server-id>

Refresh MCP tool discovery

te mcp tools list/enable/disable/toggle

Administer discovered MCP tools

Compliance Automation

Command

Description

te compliance validate --data '<json>'

Validate bounded content against adopted rulepacks

te compliance rewrite --data '<json>'

Produce and revalidate a safe rewrite

te compliance evidence <id>

Inspect compliance evidence

te compliance risks list/show/create/update

Manage the tenant risk register

te compliance risks assess/map-control/add-subject/remove-subject

Operate the risk-to-control lifecycle

te compliance source-runs create/show/submit-results/complete

Ingest normalized scanner or webhook results

te compliance certifications create/show

Run and inspect compliance certification jobs

Tenant Admin Automation

These commands require an API token for a tenant owner or tenant admin. They are designed for CI smoke tests and end-to-end product checks. Secret fields can be sent on create/update where the server supports them, but responses never print stored provider keys, AWS secrets, or invitation tokens.

Command

Description

te tenant resources

List supported tenant resource names

te tenant list <resource>

List resources such as inference_keys, inference_roles, model_deployments, routing_profiles, guardrail_policies, governance_policies, mcp_servers, tenant_agents, tenant_skills, and credential_sources

te tenant show <resource> <id>

Show one tenant resource

te tenant create <resource> --data '<json>'

Create a tenant resource from JSON

te tenant update <resource> <id> --data '<json>'

Update a tenant resource from JSON

te tenant delete <resource> <id>

Delete a tenant resource; inference keys are revoked

te tenant validate guardrail_policies --data '<json>' --sample-text 'hello'

Validate/test an unsaved simple guardrail without creating records

te tenant validate governance_policies --data '<json>' --context '<json>'

Validate/test an unsaved Governance Rule without creating records

te tenant test-policy <id> --context '<json>'

Dry-run a Governance Rule

te tenant test governance_policies <id> --context '<json>'

Compatibility alias for governance policy dry-runs

te tenant team list

List tenant members, pending invitations, and allowed domains

te tenant team invite <email> --role member

Invite a user by email; the invite token is emailed and never printed

te tenant team set-role <member-id> --inference-role-id <id>

Assign an inference role to a member

te tenant team disable <member-id>

Disable a member

te tenant team enable <member-id>

Re-enable a member

te tenant team remove <member-id>

Remove a member

te tenant team cancel-invite <invitation-id>

Cancel a pending invitation

te tenant team domains --set "example.com,example.org"

Replace allowed email domains

te tenant capture show

Show inference capture settings

te tenant capture update --data '<json>'

Update inference capture settings

Billing & Account

Command

Description

te billing show

Balance and transaction history

te billing add-credits

Open browser to add credits

te account

Account info

Configuration

Command

Description

te config set-token <key>

Set API key manually

te config set-url <url>

Override API URL

te config show

Show current config

All commands support --json for machine-readable output.

MCP Tools Reference

Training Jobs

Tool

Description

create_job

Fine-tune an LLM on a GitHub repo. Supports agent selection (Cody, SIERA), quality tier, base model, epochs, S3 export.

estimate_job

Cost estimate before training. Returns cost range, balance, sufficiency check.

list_jobs

List training jobs with status filter

show_job

Full job details including agent, model, GPU usage, cost, retry info

job_status

Live status with GPU minutes, charges, delivery progress

cancel_job

Cancel a running/queued job

retry_job

Retry a failed job from its last checkpoint

Models

Tool

Description

list_models

List trained and imported models

show_model

Model details (status, size, base model, training job)

delete_model

Delete a model from cloud storage

model_status

Import/export progress

list_supported_models

Available base models with GPU hours per epoch

Marketplace

Tool

Description

list_catalog_models

Browse pre-built models and datasets

get_catalog_model

Details of a marketplace item

catalog_export_status

Check marketplace export progress

Datasets

Tool

Description

list_datasets

List datasets for training and evaluation

show_dataset

Dataset details and status

create_dataset

Create a dataset from S3

delete_dataset

Delete a dataset

dataset_status

Check dataset import/processing status

Evaluations

Tool

Description

list_evaluations

List model evaluations

show_evaluation

Evaluation details, scores, and metrics

create_evaluation

Run an evaluation against a dataset

cancel_evaluation

Cancel a running evaluation

evaluation_status

Live evaluation progress

list_evaluators

Available evaluators (code_execution, similarity, llm_judge, etc.)

estimate_evaluation

Cost estimate for an evaluation

Inference

Tool

Description

list_inference_models

Models available for inference

inference_usage

Inference API usage statistics

get_inference_jwt

Get JWT token for direct API access

get_inference_token

Exchange an inference key for a short-lived inference JWT

call_inference

Call chat, Responses, embeddings, or Messages using the configured credential

send_agent_message

Send a governed A2A agent message

list_skills / invoke_skill

Discover, prepare, or invoke governed skills

Runtime, Policy, and Approvals

Tool

Description

list_traces

List runtime traces

show_trace

Show a trace with linked events, policy decisions, and approvals

create_trace

Ingest a trace payload without secrets

list_outcomes

List observed outcomes/goals normalized as success signals

list_insights

List Insight Loop recommendations

show_insight

Show one Insight Loop recommendation

doctor_simulate

Simulate inference access, role, endpoint, policy, and resource checks

record_outcome

Record an outcome/goal signal; requires --enable-registry-writes

map_outcome

Create an outcome mapping rule; requires --enable-registry-writes

accept_insight

Accept an insight for review; requires --enable-registry-writes

apply_insight

Apply or queue an accepted insight; requires --enable-registry-writes

list_policy_decisions

List AGT YAML policy decisions

show_policy_decision

Show one decision with redacted context

list_policy_templates

List curated AGT YAML policy templates

render_policy_template

Render disabled/shadow policy YAML from safe structured parameters

generate_policy_draft

Generate an AI-assisted disabled/shadow draft; secret-looking prompts are refused

list_approvals

List policy approval requests

show_approval

Show one approval request

approve_approval

Approve a pending request

deny_approval

Deny a pending request

list_runtime_interventions / show_runtime_intervention

Inspect runtime control requests

create/ack/complete/fail_runtime_intervention

Operate intervention lifecycle; requires --enable-registry-writes

list/show/upsert_runtime_state_reference

Manage safe external state and memory pointers

registry_sync_dry_run / registry_sync_apply / show_registry_sync

Diff, apply, and inspect registry manifests

list_work_sessions / show_work_session / complete_work_session

Inspect and complete Work Sessions

list_initiatives / show_initiative

Inspect initiative groupings

list_compliance_risks / show_compliance_risk

Inspect the tenant risk register

validate_compliance / show_compliance_evidence

Validate content and inspect evidence

create/submit/complete_compliance_source_run

Ingest normalized external test results; writes require --enable-registry-writes

show_compliance_certification

Inspect a certification run

Tenant Admin MCP Tools

These tools require a tenant owner/admin API token. The MCP server refuses internal proxy routes, inference-key creation, and raw secret-bearing mutation fields.

Tool

Description

list_tenant_resources

List allowlisted tenant resource names

tenant_resource_list

List models, roles, policies, MCP servers, agents, skills, credential sources, and related metadata

tenant_resource_show

Show one resource without returning stored secrets

tenant_resource_create

Create non-secret tenant registry/config metadata

tenant_resource_update

Update non-secret tenant registry/config metadata

tenant_resource_delete

Delete or revoke a tenant resource

tenant_resource_validate

Validate/test unsaved guardrail or AGT policy payloads without creating records

test_governance_policy

Dry-run an AGT YAML governance policy

tenant_team_list

List members, invitations, and allowed domains

tenant_team_invite

Invite a user without returning invitation tokens

tenant_team_set_inference_role

Assign or clear an inference role

tenant_team_disable / tenant_team_enable

Disable or re-enable a member

tenant_team_remove

Remove a tenant member

tenant_invitation_cancel

Cancel a pending invitation

tenant_domains_update

Replace allowed email domains

inference_capture_show / inference_capture_update

Manage request-capture settings using credential-source references

Agents

Tool

Description

list_agents

List available agents

show_agent

Agent details and capabilities

Account

Tool

Description

get_balance

Account balance and recent transactions

get_account

Account details

Environment Variables

Variable

Description

TE_API_KEY

API key (overrides config file)

TE_API_URL

API URL (default: https://app.tuningengines.com)

TE_INFERENCE_KEY

Optional sk-te-* key for direct model, agent, skill, and MCP calls

TE_INFERENCE_URL

Inference base URL (default: https://api.tuningengines.com/v1)

Tenant management commands keep the configured te_* API token local and exchange it for a short-lived management JWT before calling the API. Inference keys (sk-te-*) are for inference-only flows such as te inference token and proxy calls; they are not accepted for tenant registry management commands.

Inference Smoke Testing

Use te-inference-smoke to exercise inference behavior as a tenant admin and, optionally, real tenant users. The default run is read-only. Set TE_SMOKE_MUTATE=1 to create temporary inference roles, keys, policies, guardrails, MCP servers, agents, and skills, then test permission permutations and clean them up.

If you only have an sk-te-* inference key, set TE_INFERENCE_KEY for proxy-only checks. Full role/user/policy permutations require a tenant-admin app API key that starts with te_.

TE_API_URL=https://app.tuningengines.com \
TE_ADMIN_API_KEY=te_admin_key_here \
TE_USER_API_KEY=te_user_key_here \
npx -y --package tuningengines-cli@latest te-inference-smoke

For actual proxy model calls, enable live calls explicitly:

TE_API_URL=https://app.tuningengines.com \
TE_INFERENCE_BASE=https://api.tuningengines.com/v1 \
TE_ADMIN_API_KEY=te_admin_key_here \
TE_SMOKE_MUTATE=1 \
TE_SMOKE_LIVE_CALLS=1 \
TE_SMOKE_CREATE_MODEL_DEPLOYMENT=1 \
TE_SMOKE_ALLOWED_MODEL=llama-3.1-8b-fast \
TE_SMOKE_DENIED_MODEL=llama-3.3-70b-fp8 \
TE_SMOKE_AGENT_URL=https://httpbin.org/post \
npx -y --package tuningengines-cli@latest te-inference-smoke

TE_SMOKE_CREATE_MODEL_DEPLOYMENT=1 is useful for disposable tenants that do not already have an enabled model. By default the runner treats a provider authentication failure on an allowed model as proof that Tuning Engines RBAC allowed the request through to the provider. Set TE_SMOKE_ALLOW_PROVIDER_AUTH_FAILURE=0 when the tenant has real provider credentials and the allowed call must return 200.

To test multiple tenant users, provide their API tokens:

TE_SMOKE_USERS_JSON='[
  {"email":"member1@example.com","api_key":"te_user_key_1"},
  {"email":"member2@example.com","api_key":"te_user_key_2"}
]' \
TE_ADMIN_API_KEY=te_admin_key_here \
TE_SMOKE_MUTATE=1 \
npx -y --package tuningengines-cli@latest te-inference-smoke

Preview coverage:

npx -y --package tuningengines-cli@latest te-inference-smoke --list

Each run writes a masked JSON report under te-smoke-results/, or to TE_SMOKE_REPORT when that env var is set.

Authentication

te auth login uses a secure device authorization flow (same pattern as gh auth login):

  1. CLI generates a device code and opens your browser

  2. Sign up or log in (email/password, Google, or GitHub)

  3. Click "Authorize" to grant CLI access

  4. Token flows back automatically — no copy-paste

Works for both new sign-ups and existing accounts. Token saved to ~/.tuningengines/config.json with 0600 permissions.

License

MIT

Available Tools

38 tools
cancel_evaluationB

Cancel a running or queued evaluation.

ParametersJSON Schema
NameRequiredDescriptionDefault
evaluation_idYesEvaluation ID to cancel

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It only states 'Cancel', which implies mutation, but does not specify if the action is irreversible, any side effects, or what happens to the evaluation's data and associated resources.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no extraneous words. It efficiently communicates the core function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple cancel action with one parameter and no output schema, the description is minimally adequate. However, it lacks information about error conditions, typical use cases, or confirmation of state change, which would be useful for an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides a description for the single parameter 'evaluation_id' ('Evaluation ID to cancel'). The tool description adds no additional meaning beyond what is in the schema, so baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Cancel' and the resource 'evaluation', and specifies the applicable states 'running or queued', which prevents misuse on completed evaluations. Although a sibling tool 'cancel_job' exists, the resource differentiation is clear from the tool name itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like 'cancel_job' or 'delete_evaluation'. There is no mention of prerequisites, such as checking the evaluation status first, or that cancellation may not be possible beyond a certain point.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cancel_jobA

Cancel a running or queued fine-tuning job. The job will be charged for any GPU time already used.

ParametersJSON Schema
NameRequiredDescriptionDefault
job_idYesJob ID to cancel

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses a key behavioral trait beyond annotations: the job will be charged for GPU time already used. This adds significant value and helps the agent understand the financial impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, and no wasted words. Every sentence serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (1 parameter, no output schema), the description covers the purpose, parameter, and a key consequence (charging). It is fully adequate for this context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and description of parameter (job_id) is already provided in the schema. The description adds no further semantic information beyond what the schema indicates.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Cancel') and the specific resource ('running or queued fine-tuning job'), making it distinct from siblings like cancel_evaluation or retry_job.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when canceling a fine-tuning job but does not explicitly state when not to use it or provide alternatives. The context of sibling tools offers implicit differentiation, but explicit guidance is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

catalog_export_statusA

Check the status of a Marketplace export operation. Returns status, charge info, and any error messages.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_idYesCatalog model ID (UUID)
export_idYesExport operation ID (UUID)

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the tool returns 'status, charge info, and any error messages', which gives some behavioral insight. However, it does not specify whether the operation is read-only, idempotent, or requires authentication, leaving gaps in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with two short sentences: the first states the purpose, the second lists what is returned. Every word is necessary, and no fluff is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (two required UUID parameters, no output schema), the description covers the basic return values. However, it lacks context such as the need to call export_catalog_model first, whether polling is safe, or how to interpret errors. It is adequate but not fully comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the base score is 3. The description adds no additional semantic information about the parameters (model_id and export_id) beyond what the schema already provides. It does not clarify their format or relationship beyond UUIDs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Check the status of a Marketplace export operation', specifying the verb (check), resource (status of export operation), and scope (Marketplace). It distinguishes this tool from sibling status tools like job_status or evaluation_status by being specific to Marketplace export operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus alternatives or mention prerequisites. While it's implied that it should be used after initiating an export via export_catalog_model, there is no direct guidance. The name helps, but explicit usage context would improve clarity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_datasetA

Create a new dataset by importing from S3. Datasets can be used for fine-tuning or model evaluation.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName for the dataset
descriptionNoDescription of the dataset contents
source_typeYesSource type (e.g. 's3')
s3_urlNoS3 URL of the dataset (e.g. s3://bucket/path/data.jsonl)
s3_access_key_idNoAWS access key ID
s3_secret_access_keyNoAWS secret access key
s3_regionNoAWS region (e.g. us-east-1)
for_evaluationNoWhether this dataset is for evaluation (default: false)

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are absent, so the description must disclose all behavioral traits. It mentions the S3 import mechanism but fails to detail failure modes, error handling, idempotency, or permission requirements (e.g., AWS credentials, region). The impact of existing datasets with the same name is not addressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. Action is front-loaded ('Create a new dataset'), and the purpose is clearly stated. Ideal conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 parameters, no output schema, and no annotations, the description provides acceptable context (S3 import, usage for fine-tuning/evaluation) but lacks critical details like error handling, duplicate handling, and output expectations for a creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds no additional meaning beyond the schema; it merely restates the S3 import aspect. No further enrichment of parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create a new dataset'), the source ('importing from S3'), and the purpose ('fine-tuning or model evaluation'). It effectively distinguishes from sibling tools like 'delete_dataset' or 'list_datasets'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives. The description does not mention prerequisites, when not to use it, or suggest other tools like 'validate_s3' for pre-checking S3 access.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_evaluationA

Create a new model evaluation. Run your trained model or a base model against a dataset using selected evaluators. Use list_evaluators to see available evaluators (e.g. code_execution, similarity, llm_judge).

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoName for this evaluation run
user_model_idNoID of your trained model to evaluate. Either this or base_model is required.
base_modelNoHuggingFace model ID to evaluate (e.g. 'Qwen/Qwen2.5-Coder-7B-Instruct'). Either this or user_model_id is required.
dataset_idYesID of the evaluation dataset to use. Must be a dataset marked for_evaluation.
evaluator_idsYesList of evaluator IDs to run (use list_evaluators to see options)
max_samplesNoMaximum samples to evaluate (default: all)

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must cover behavioral traits. It implies a run action but doesn't disclose side effects like cost, asynchronous behavior, or resource usage. The description is too brief for a tool with no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, with the purpose front-loaded. Every sentence adds meaningful information without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the schema covers all parameters and there is no output schema, the description provides an adequate but minimal overview. It could be more complete by noting return values or asynchronous behavior, but it suffices for a straightforward creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes all parameters with 100% coverage. The description adds value by referencing list_evaluators for available evaluators and clarifying that either user_model_id or base_model is required (though this contradicts the schema's optionality). This extra context helps the agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates a new model evaluation and explains the process (running a model against a dataset with evaluators). It distinguishes itself from sibling tools like list_evaluators and show_evaluation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions using list_evaluators to find available evaluators, which provides some guidance. However, it doesn't explicitly state when not to use this tool, such as when to use estimate_evaluation instead, or clarify the choice between user_model_id and base_model.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_jobA

Fine-tune an LLM on a GitHub repository using Tuning Engines. This trains a custom model that learns from the code patterns, style, and conventions in the repo. Choose an agent to control the training approach:

AVAILABLE AGENTS:

  • agent='code_repo' (Cody) — LoRA-based code fine-tuning using QLoRA (4-bit quantized LoRA) via the Axolotl framework. Trains on your repo's code patterns, naming conventions, and project structure to produce a fast, lightweight adapter. Best for: code autocomplete, inline suggestions, tab-complete, code style matching.

  • agent='sera_code_repo' (SIERA) — Bug-fix specialist using the Open Coding Agents approach from AllenAI. Generates synthetic error-resolution training pairs from your repo, producing a model that understands your codebase's failure patterns and fix conventions. Best for: debugging, error resolution, patch generation, root cause analysis. Supports quality_tier='low' (faster) or quality_tier='high' (deeper analysis, more training data).

SUPPORTED BASE MODELS (by size):

  • 3B: Qwen/Qwen2.5-Coder-3B-Instruct

  • 7-8B: codellama/CodeLlama-7b-hf, deepseek-ai/deepseek-coder-7b-instruct-v1.5, Qwen/Qwen2.5-Coder-7B-Instruct, Qwen/Qwen3-8B

  • 13-15B: codellama/CodeLlama-13b-Instruct-hf, bigcode/starcoder2-15b, Qwen/Qwen2.5-Coder-14B-Instruct, Qwen/Qwen3-14B

  • 22-27B: mistralai/Codestral-22B-v0.1, google/gemma-2-27b

  • 30-34B: deepseek-ai/deepseek-coder-33b-instruct, codellama/CodeLlama-34b-Instruct-hf, Qwen/Qwen2.5-Coder-32B-Instruct, Qwen/Qwen3-Coder-30B-A3B, Qwen/Qwen3-32B

  • 70-72B: codellama/CodeLlama-70b-Instruct-hf, meta-llama/Llama-3.1-70B-Instruct, Qwen/Qwen2.5-72B-Instruct

TYPICAL WORKFLOW: estimate_job first to check cost, then create_job, then job_status to monitor progress.

ParametersJSON Schema
NameRequiredDescriptionDefault
base_modelNoHuggingFace model ID to fine-tune (e.g. 'Qwen/Qwen2.5-Coder-7B-Instruct'). Required unless base_user_model_id is provided. Use list_supported_models to see all options.
base_user_model_idNoID of a previously trained model to fine-tune further (iterative training). The base model is resolved automatically. Use list_models to find IDs.
output_nameYesName for the resulting fine-tuned model (e.g. 'my-project-cody-7b')
repo_urlYesGitHub repository URL to train on (e.g. 'https://github.com/org/repo')
branchNoGit branch to use (default: main)
num_epochsNoNumber of training epochs (more = better quality but higher cost)
max_examplesNoMaximum training examples to extract from the repo (minimum: 2)
agentNoTraining agent to use. 'code_repo' (Cody) = QLoRA-based fine-tuning for code autocomplete and inline suggestions. 'sera_code_repo' (SIERA) = bug-fix specialist using AllenAI's Open Coding Agents approach. Default: 'code_repo'.
quality_tierNoQuality tier (SIERA agent only). 'low' = faster, fewer synthetic pairs. 'high' = deeper analysis, more training data, better results. Default: 'low'.
s3_output_bucketNoS3 bucket to export the trained model to. If omitted, model is stored in Tuning Engines cloud storage.
s3_access_key_idNoAWS access key ID for S3 export
s3_secret_access_keyNoAWS secret access key for S3 export
s3_regionNoAWS region for S3 export (e.g. us-east-1)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It explains that training produces a custom model using QLoRA (code_repo) or synthetic error-resolution pairs (sera_code_repo), and mentions that the model can be stored in cloud storage or exported to S3. It does not detail potential side effects, rate limits, or the exact output format, but the description is fairly transparent about the training process and output destinations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with sections for agents, base models, and workflow, but it is quite lengthy, particularly the exhaustive list of supported base models which could be omitted since users can use list_supported_models. While the structure is clear, conciseness is slightly compromised by the inclusion of redundant details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of fine-tuning an LLM, the description is fairly complete. It explains the tool's purpose, agents, supported base models, and workflow. It references sibling tools for cost estimation and monitoring. However, it does not describe the return value or response structure (e.g., job ID or status), which would be helpful. Since there is no output schema, a brief note on what the tool returns would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so the schema already documents all parameters. The description adds value by providing extra context for the agent and quality_tier parameters, including best-use recommendations ('Best for: code autocomplete, inline suggestions...') and agent-specific behavior. This goes beyond the schema's enum descriptions, justifying a score above the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Fine-tune an LLM on a GitHub repository using Tuning Engines.' It specifies the action (fine-tune), the resource (LLM), and the data source (GitHub repo). The description distinguishes the tool from siblings like estimate_job and job_status by outlining the typical workflow, and it further differentiates the two agents (code_repo and sera_code_repo) with explicit use cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidelines, including a 'TYPICAL WORKFLOW' section that advises users to run estimate_job first for cost estimation, then create_job, and finally job_status to monitor progress. It also clarifies when to use each agent and the quality_tier parameter for the SIERA agent, offering clear decision criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

dataset_statusC

Check the status of a dataset import or processing operation.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataset_idYesDataset ID (UUID)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose any behavioral traits such as side effects, authorization requirements, or rate limits. As a read-only status check, more detail on safety and repeatability would be helpful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence with no wasted words. It is appropriately concise for a simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity tool with one parameter and no output schema, the description is adequate but could be improved by indicating possible return values or statuses.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (one parameter with description). The description adds no extra meaning beyond the schema; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'check' and the resource 'status of a dataset import or processing operation'. It is specific to datasets, distinguishing it from other status tools like job_status, but does not explicitly differentiate from show_dataset.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like show_dataset or job_status. There is no mention of prerequisites or context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_datasetC

Delete a dataset from the platform.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataset_idYesDataset ID to delete

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description bears full responsibility for behavioral disclosure. It only states 'delete' without indicating that the operation is destructive, irreversible, or whether it affects dependent resources. Important behavioral traits are missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no wasted words. It is front-loaded with the verb and resource. However, it could include a few more critical details without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although the tool is simple with one parameter, the description fails to mention essential context like whether the deletion is permanent, any prerequisites (e.g., ownership), or side effects. This lack of completeness could lead to misuse.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a description for dataset_id. The tool description adds no additional meaning beyond what the schema already provides. Baseline 3 is appropriate as the schema carries the full burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (delete a dataset) and the resource (dataset from the platform). It effectively distinguishes from sibling tools like create_dataset (create) and show_dataset (read). However, it could be more specific by noting the deletion is permanent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. It does not mention prerequisites, irreversibility, or when deletion should be avoided. This lack of context makes it harder for an agent to decide appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_modelB

Delete a trained model from cloud storage.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_idYesModel ID to delete

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry full burden. It indicates a destructive action but lacks details on side effects (e.g., cascading deletes, reversibility, permission requirements). The 'delete' verb implies destruction, but more context is needed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that efficiently conveys the tool's purpose without extraneous words. It is front-loaded with the key action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete operation with one required parameter and no output schema, the description adequately covers the basic action. It could mention permanence or prerequisites, but overall it is sufficient given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with one parameter 'model_id' described as 'Model ID to delete'. The description adds no extra meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'delete' and resource 'trained model' with location 'from cloud storage'. It distinguishes itself from sibling tools like 'delete_dataset' by specifying the resource type, providing clear purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'cancel_evaluation' or 'delete_dataset'. There are no usage conditions, prerequisites, or notes about when deletion is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

estimate_evaluationB

Get a cost estimate for an evaluation before running it.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_model_idNoID of your trained model
base_modelNoOr a HuggingFace model ID
dataset_idYesEvaluation dataset ID
evaluator_idsYesList of evaluator IDs
max_samplesNoMax samples to evaluate

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full responsibility for behavioral disclosure. The description only states the tool returns a cost estimate, but fails to mention any behavioral traits: it does not indicate whether the operation is read-only, whether it affects state, what the response format is, or if authentication or quota are needed. This is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no redundant words. It front-loads the core purpose, making it immediately clear what the tool does. Every part of the description earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (5 parameters, no output schema, no annotations), the description is too sparse. It does not explain what the cost estimate depends on (e.g., model, dataset size, evaluators), how to interpret the result, or any constraints (e.g., balance requirement). The description lacks completeness for a meaningful cost estimation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema describes all 5 parameters with descriptions (100% coverage), so the baseline is 3. The description adds no additional parameter semantics beyond what the schema already provides. It does not explain how each parameter influences the cost estimate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get a cost estimate for an evaluation before running it.' It specifies a concrete verb ('Get') and resource ('cost estimate for evaluation'), effectively distinguishing it from sibling tools such as create_evaluation (which runs the evaluation) and estimate_job (which estimates jobs, not evaluations).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage before running an evaluation ('before running it'), providing some context. However, it does not explicitly state when to use this tool over alternatives, nor does it mention prerequisites or when not to use it. The guidance is implicit rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

estimate_jobA

Get a cost estimate for a fine-tuning job before submitting it. Returns estimated cost, cost range, current balance, and whether balance is sufficient. Always estimate before creating a job.

ParametersJSON Schema
NameRequiredDescriptionDefault
base_modelNoHuggingFace model ID (e.g. 'Qwen/Qwen2.5-Coder-7B-Instruct'). Required unless base_user_model_id is provided.
base_user_model_idNoID of a previously trained model. The base model is resolved automatically.
num_epochsNoTraining epochs
max_examplesNoMaximum examples
repo_size_mbNoApproximate repository size in MB (helps refine the estimate)
use_caseNoAgent to use for the estimate (e.g. 'code_repo' for Cody, 'sera_code_repo' for SIERA). Defaults to code_repo.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description fully bears responsibility. It discloses return information (estimated cost, cost range, current balance, sufficiency). It does not mention side effects (likely none) or authorization, but the read-only nature is implied.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise sentences with no redundant information. It front-loads the core purpose and includes a directive.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 6 parameters and no output schema, the description could be more complete by specifying the return format or units. It lists return items but lacks detail like how errors are handled.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds no additional parameter meaning beyond what the schema already provides (e.g., 'HuggingFace model ID').

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get a cost estimate'), the resource ('a fine-tuning job'), and the purpose ('before submitting it'). It distinguishes from sibling tools like estimate_evaluation by specifying the job type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises 'Always estimate before creating a job,' providing clear when-to-use guidance. It does not specify when not to use, but the context implies it's a prerequisite step.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evaluation_statusB

Get live status of an evaluation including progress and current metrics.

ParametersJSON Schema
NameRequiredDescriptionDefault
evaluation_idYesEvaluation ID (UUID)

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It mentions 'live' status but does not describe side effects, authorization needs, latency, or consistency. Read-only is implied but not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence. It is efficient without being overly terse, and it front-loads the key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description gives a high-level idea of the output (progress and metrics) but no detail on the structure or data fields. Given no output schema, more context would be helpful for an agent to interpret results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter evaluation_id is fully documented in the schema (100% coverage). The tool description adds no additional meaning to the parameter beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get'), the resource ('live status of an evaluation'), and what it includes ('progress and current metrics'). It distinguishes from sibling tools like list_evaluations by focusing on live, individual status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like show_evaluation or job_status. Only the purpose is stated, without contextual usage advice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

export_catalog_modelA

Export a pre-built model or dataset from the Marketplace to your S3 bucket. Credits will be charged based on the export price upon successful completion.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_idYesCatalog model ID (UUID) to export
s3_bucketYesDestination S3 bucket name
s3_prefixNoOptional S3 key prefix for the exported model
s3_access_key_idYesAWS access key ID
s3_secret_access_keyYesAWS secret access key
s3_regionYesAWS region (e.g. us-east-1)

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided; description only adds credits charge. Lacks disclosure of async nature, side effects on S3, or failure behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences conveying purpose and cost without unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Missing important context: async operation (indicated by sibling catalog_export_status), no mention of return value, no permissions guidance for S3 credentials.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters with descriptions. The tool description adds no additional parameter context beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action (export) and resource (pre-built model or dataset from Marketplace to S3), distinguishing it from sibling tools like export_model.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies it's for marketplace exports and mentions cost, but does not explicitly compare with alternatives or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

export_modelA

Export a trained model from Tuning Engines cloud storage to your S3 bucket.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_idYesModel ID (UUID) to export
s3_bucketYesDestination S3 bucket name
s3_prefixNoOptional S3 key prefix for the exported model
s3_access_key_idYesAWS access key ID
s3_secret_access_keyYesAWS secret access key
s3_regionYesAWS region (e.g. us-east-1)
delete_afterNoDelete the model from Tuning Engines storage after export (default: false)

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It only states the action but does not disclose side effects (e.g., whether the original is deleted by default, if it's synchronous or async, or required permissions beyond schema). This is minimal for a mutation tool with no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence (12 words) that front-loads the purpose. Every word is necessary, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite a rich schema, the description lacks important context: no mention of typical workflow (e.g., verifying export status), no explanation of the 'delete_after' parameter behavior, and no guidance on prerequisites. Incomplete for a 7-parameter tool with no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 7 parameters have descriptions in the input schema (100% coverage). The description adds no additional parameter-specific information beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Export') and the resource ('trained model from Tuning Engines cloud storage to your S3 bucket'). It distinguishes itself from siblings like 'export_catalog_model' by specifying the source and destination.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use the tool (export trained model to S3). It does not explicitly state when not to use it or mention alternatives, but the context is sufficient for an AI agent to infer appropriate usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_accountA

Get your Tuning Engines account details and settings.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. The description only says 'Get', implying no side effects, but does not disclose what exactly is returned, authentication requirements, or any limitations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence with no unnecessary words. Front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema and no annotations, the description is minimal but adequate. It lacks detail on what fields or settings are returned.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, and schema coverage is 100% (empty object). Per guidelines, baseline for 0 params is 4, and description adds no additional meaning but is not needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'your Tuning Engines account details and settings.' It is distinct from sibling tools like get_balance and get_catalog_model.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use or not use this tool. It does not mention alternatives or contexts where it is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_balanceB

Check your Tuning Engines account balance and recent transactions.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so the description carries the full burden. It indicates a read operation ('Check') but does not disclose behavioral traits such as authentication needs, rate limits, or what 'recent' means. Basic safety is implied but not detailed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence with no unnecessary words. Extremely concise and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters, no output schema, and no annotations, the description covers the basic purpose. However, it lacks specifics about the output format, transaction count, or time range for 'recent'. Somewhat incomplete for a well-rounded understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so schema coverage is 100%. The description adds no parameter info, which is acceptable since there are none. Baseline 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool checks 'account balance and recent transactions', which is a specific verb+resource. However, it does not differentiate from the sibling tool 'get_account', which might also provide account details.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool vs. alternatives like 'get_account'. The description implies it's for checking balance and transactions, but does not explicitly state context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_catalog_modelA

Get detailed information about a specific pre-built model or dataset from the Marketplace including description, pricing, and export options.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_idYesCatalog model ID (UUID)

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavioral traits. While 'Get' implies a read operation, it does not explicitly state that the tool is read-only, safe to call without side effects, or any prerequisites like authentication or quota implications. This leaves ambiguity for the agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-formed sentence that conveys the purpose and key details without redundancy or unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple lookup tool with one parameter and no output schema, the description covers the return value (description, pricing, export options) adequately. It could mention that the response includes all available metadata, but it is sufficiently complete for most use cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage describing the single parameter 'model_id' as a UUID. The description adds no extra meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the verb 'Get' and specifies the resource as 'a specific pre-built model or dataset from the Marketplace', clearly indicating a singular retrieval operation. This distinguishes it from sibling tools like 'list_catalog_models' (listing) and 'export_catalog_model' (exporting).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives such as 'list_catalog_models' for browsing or 'catalog_export_status' for export tracking. The description implies usage when needing detailed info on a single item, but lacks exclusions or when-not scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_inference_jwtA

Get a JWT token for authenticating with the Tuning Engines inference API. Use this to make direct API calls to the inference endpoint.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries burden. States it returns a JWT token, implying a read operation, but doesn't disclose expiration, permissions, or side effects. Adequate but not detailed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, directly front-loaded with core action and purpose. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, description explains what it does and when to use it. Could note token expiration or security, but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so baseline 4. Description adds no parameter info, but schema coverage is 100%.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it retrieves a JWT token for the Tuning Engines inference API, distinct from sibling tools like get_account or inference_usage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use for direct API calls to inference endpoint, providing clear context. No exclusions or alternatives mentioned, but purpose is straightforward.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

import_modelA

Import a model from S3 into Tuning Engines cloud storage so it can be used as a base for future fine-tuning jobs.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName for the imported model
source_s3_urlYesS3 URL of the model to import (e.g. s3://bucket/path/to/model)
base_modelYesHuggingFace model ID that this model was fine-tuned from
s3_access_key_idYesAWS access key ID
s3_secret_access_keyYesAWS secret access key
s3_regionYesAWS region (e.g. us-east-1)

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description reveals that the operation is mutative and requires S3 credentials, but fails to disclose important behaviors like whether it is synchronous, what happens on duplicate names, or any permissions needed. Given no annotations, this leaves gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that immediately conveys the core purpose. No redundant or extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutative tool with 6 required parameters and no annotations, the description is minimal. It explains the 'what' and 'why' but not the expected outcome, error states, or side effects. Adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All six parameters are documented in the input schema with adequate descriptions. The tool description adds no extra parameter-level context beyond the schema, so score is baseline 3 due to full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('import a model from S3'), the destination ('into Tuning Engines cloud storage'), and the purpose ('so it can be used as a base for future fine-tuning jobs'). This distinguishes it from siblings like export_model, delete_model, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool over alternatives, nor are there any prerequisites or exclusions mentioned. The agent has no help deciding between import and other tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

inference_usageB

Get inference API usage statistics including request counts, token usage, and costs.

ParametersJSON Schema
NameRequiredDescriptionDefault
start_dateNoStart date (YYYY-MM-DD)
end_dateNoEnd date (YYYY-MM-DD)
modelNoFilter by model name

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose behavioral traits such as read-only nature, authentication requirements, rate limits, or side effects. It only describes the basic function.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that gets straight to the point without any extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description partially compensates for the lack of output schema by listing included metrics. However, it omits details like whether dates are required, response format, or aggregation behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage, so baseline is 3. The description does not add meaning beyond the schema; it mentions output contents but not parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it retrieves inference API usage statistics and lists included metrics (request counts, token usage, costs). However, it does not differentiate from sibling tools like get_catalog_model or get_inference_jwt, but the purpose is specific enough.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. The description merely implies usage without stating prerequisites, limitations, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

job_statusA

Get live status of a fine-tuning job including current status, GPU minutes used, estimated charges, remaining balance, and delivery progress. Use this to monitor a running job.

ParametersJSON Schema
NameRequiredDescriptionDefault
job_idYesJob ID

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses the tool returns live status and specific fields, but lacks details on read-only nature, error behavior, or rate limits. The disclosure is adequate but not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no wasted words. The first sentence states purpose and output, the second provides usage context. Well front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description partially describes return fields (GPU minutes, charges, etc.). For a one-parameter tool, this is fairly complete, though a full output description would be better.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% coverage with a single parameter 'job_id' described as 'Job ID'. The description adds no additional meaning beyond this, so it meets the baseline but does not elevate understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves live status of a fine-tuning job, listing specific fields like GPU minutes, charges, balance, and delivery progress. It uses a specific verb and resource, distinguishing it from sibling status tools like evaluation_status or dataset_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this to monitor a running job', providing clear when-to-use guidance. However, it does not mention when not to use it or list alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_agentsA

List available agents configured for your organization. Agents are AI assistants with specific capabilities and tool access.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states that it lists agents, but does not mention any potential side effects, permission requirements, pagination, ordering, or limits. Since this is a list operation, the agent might expect complete results, but there is no confirmation or caveat.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, with the first sentence stating the core function and the second adding helpful context about agents. Every word earns its place; there is no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, no-output-schema tool, the description is reasonably complete: it states what the tool does and defines agents. However, it lacks any mention of what information is returned for each agent (e.g., name, capabilities), which could be relevant for an agent deciding whether to use this tool. A bit more detail would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, so the input schema coverage is trivially 100%. Per guidelines, 0 parameters yields a baseline of 4. The description does not add parameter information because none exist, so it is adequate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List' and resource 'available agents', and distinguishes the tool's scope by specifying 'configured for your organization'. It also provides a brief definition of agents, which helps the agent understand what it is listing. This clearly differentiates it from sibling tools like list_models or list_datasets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide any guidance on when to use this tool versus alternatives. Sibling tools include show_agent, which likely retrieves details of a single agent, but no comparison or usage context is given. The agent is left to infer when to use list_agents versus other list or show tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_catalog_modelsA

List available pre-built models and datasets from the Tuning Engines Marketplace. These are platform-owned, ready-to-use assets that can be exported to your S3 bucket. Returns name, description, base model, size, export price, and category.

ParametersJSON Schema
NameRequiredDescriptionDefault
categoryNoFilter by category (e.g. 'code', 'bug-fix', 'general'). Omit to see all.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but description adequately explains the read-only list operation and return fields. Lacks mention of pagination or empty results, but acceptable for a simple list.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no unnecessary words; clear and front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, parameter, and return fields; no output schema needed. Missing pagination info but adequate for a straightforward list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description adds examples ('code', 'bug-fix', 'general') for the category parameter, providing extra meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it lists pre-built models/datasets from Marketplace, distinguishes from sibling list_models (user models) and list_inference_models by specifying platform-owned, ready-to-use assets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Describes the tool's purpose but does not explicitly state when to use vs alternatives; however, mentioning 'pre-built models from Tuning Engines Marketplace' implies context for selecting this tool over others.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_datasetsB

List datasets available for training and evaluation. Datasets can be uploaded from S3 and used for fine-tuning or model evaluation.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax results (default 20)

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

As a read-only listing tool, behavioral transparency is adequate. However, with no annotations, the description could mention side effects or authorization, but the simple nature of listing mitigates this gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise at two sentences, but the second sentence about S3 uploads is off-topic for a listing tool, slightly reducing relevance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one parameter, the description is mostly complete. However, it doesn't explain the return format or pagination, which is a gap without an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'limit' is fully described in the schema (100% coverage). The description adds no extra meaning beyond that, so baseline score is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool lists datasets for training and evaluation. While it doesn't explicitly differentiate from siblings like 'show_dataset', the name and description make the purpose obvious.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'show_dataset' or 'create_dataset'. The second sentence about uploading S3 data is irrelevant to listing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_evaluationsB

List model evaluations. Evaluations run your trained models against benchmark datasets using various evaluators to measure quality.

ParametersJSON Schema
NameRequiredDescriptionDefault
statusNoFilter by status: queued, running, succeeded, failed, canceled
limitNoMax results (default 20)

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states the basic purpose and does not disclose any behavioral traits such as pagination, default ordering, or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: first sentence states the action, second provides context. It is concise but the second sentence is explanatory rather than directly about the tool's usage, which is acceptable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers what the tool does and the context of evaluations. However, it lacks information about the output structure (e.g., what fields are returned) since there is no output schema. For a simple list tool, it is minimally adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 2 parameters with 100% description coverage. The description does not add any meaning beyond the schema; it doesn't mention the parameters. Baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List model evaluations' with a specific verb and resource, and the additional sentence explains what evaluations are, distinguishing it from related tools like create_evaluation or evaluation_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for listing evaluations but provides no explicit guidance on when to use it versus alternatives such as list_datasets or list_models. No exclusions or context are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_evaluatorsA

List available evaluators for model evaluation. Evaluators measure different aspects of model quality like code execution, similarity, or LLM-based judgment.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only describes the purpose of evaluators, not the tool's behavior (e.g., read-only, auth requirements, response format). This is insufficient for a mutation-free listing tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences front-load the purpose and provide context about evaluator types. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description could explain what the output represents (e.g., list of evaluator IDs/names). It adequately covers the basic purpose but lacks completeness regarding output structure and differentiation from similar list tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero properties, so the description cannot add parameter meaning. Baseline score of 4 is appropriate since no parameters require explanation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List' and the resource 'available evaluators for model evaluation', distinguishing it from siblings like 'list_evaluations' which likely lists evaluation jobs. It also provides examples of evaluator types, adding specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like list_evaluations or list_models. The description implies usage when selecting evaluators for a model evaluation, but lacks when-not conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_inference_modelsA

List models available for inference through the Tuning Engines inference API. Includes both platform models and your deployed trained models.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided (no readOnlyHint). The description accurately states it lists models, which implies read-only behavior. However, it lacks details on performance, pagination, or any potential side effects, which is acceptable for a simple 0-param tool but could be more transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences that front-load the purpose and include key details about what is listed. No redundant phrases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 0-param tool with no output schema, the description is sufficient. It explains the scope (platform + trained models) and the API context. Could mention that output is a list of model IDs or similar, but not necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 0 parameters, so description does not need to elaborate on params. Baseline 4 is appropriate as the description adds no param info but schema coverage is 100% and no params exist.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists models for inference via Tuning Engines API, distinguishing it from generic model listing tools like `list_models` by specifying the inference context and included model types (platform + deployed trained).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when needing inference-available models, but does not explicitly state when not to use or provide comparisons to siblings like `list_models` or `list_catalog_models`. The context is clear enough for a simple tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_jobsA

List fine-tuning training jobs on Tuning Engines. Returns recent jobs with status, base model, agent type, GPU usage, and cost. Use this to check on existing training runs or find a job ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
statusNoFilter by status: queued, running, succeeded, failed, canceled
limitNoMax results (default 20)

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so description must disclose behavior. It indicates a read-only listing operation with no destructive hints, which is accurate. It does not address potential rate limits or pagination, but the basic behavior is clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: one states the action and resource, the other states return fields and use case. Every sentence is informative with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, return fields, and usage context. Missing details like ordering or permissions are minor for this simple list tool, but could be slightly more complete by referencing sibling tools for specific job details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Parameter schema coverage is 100% with clear descriptions for status and limit. The description does not add new meaning beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists fine-tuning training jobs and specifies the returned fields (status, model, agent, GPU, cost). It differentiates from sibling listing tools like list_agents or list_datasets by focusing on jobs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description tells when to use it (check existing runs, find job ID). However, it does not explicitly contrast with related tools like job_status, though the context implies a browsing use case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_modelsB

List your trained and imported models on Tuning Engines.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description should disclose behavioral traits. It only says 'list' and does not mention whether it's read-only, response format, pagination, or any side effects. The read-only nature is implied but not confirmed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, front-loaded, with no wasted words. It is optimally concise for the information provided.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no parameters, the description is incomplete. It does not explain what information is returned (e.g., model IDs, names, statuses) or any filtering capabilities, which is necessary for an agent to use it effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, so by rule baseline is 4. The description does not add parameter details because none exist.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists trained and imported models on Tuning Engines. It distinguishes from sibling listing tools like list_catalog_models by specifying 'your' models, but could be more explicit about the scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives (e.g., list_catalog_models, list_supported_models). It only states the basic function without context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_supported_modelsA

List the supported base HuggingFace models available for fine-tuning on Tuning Engines. Optionally filter by agent to see only compatible models.

ParametersJSON Schema
NameRequiredDescriptionDefault
agentNoFilter models compatible with this agent (e.g. 'code_repo', 'sera_code_repo'). Omit to see all models.

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description correctly implies a read-only operation via 'List', but does not elaborate on any other behavior such as pagination or rate limits. Adequate for a simple list tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words, front-loaded with purpose. Excellent conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core functionality and optional filter, but does not mention return format or any output details. Still sufficient for selection given no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description adds the detail 'to see only compatible models', which provides more context than the schema description alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List', the resource 'supported base HuggingFace models', and the context 'for fine-tuning on Tuning Engines', distinguishing it from siblings like list_models or list_catalog_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions optional filtering by agent, providing context on when to use the parameter, but does not specify when not to use this tool or compare it to alternatives like list_models.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

model_statusB

Check the status of a model import or export operation.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_idYesModel ID (UUID)

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Description indicates a read-only operation ('check the status'), which is straightforward. No annotations are present, so the description does not contradict any. However, it lacks details on behavior like idempotency, error cases, or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, front-loaded with the action and resource. It is concise, though more details could be added without excessive length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no output schema and simple parameters, the description does not explain return values (e.g., status types, format). For a single-parameter tool, it could be more complete regarding what the status check entails.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema includes one parameter 'model_id' with description 'Model ID (UUID)'. The description does not add extra semantic meaning beyond the schema, and schema coverage is 100%, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Check the status of a model import or export operation.' It uses a specific verb and resource, but does not differentiate from similar sibling tools like 'catalog_export_status' or 'show_model', which may also relate to import/export status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives such as 'catalog_export_status' or 'show_model'. No context on prerequisites or typical usage scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

retry_jobA

Retry a failed fine-tuning job from its last checkpoint. Creates a new job that resumes training where the failed one stopped, saving GPU time. Each retry is billed separately.

IMPORTANT: This tool fetches a cost estimate and includes it in the response. You MUST show the estimate to the user and get their explicit approval before considering the retry confirmed. The retry is submitted automatically (the server validates balance), but always present the cost to the user.

ParametersJSON Schema
NameRequiredDescriptionDefault
job_idYesID of the failed job to retry
github_tokenNoGitHub Personal Access Token (required if original job used a private repo). Not stored — only sent to the training backend.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses key behaviors: creates a new job, bills separately, fetches a cost estimate, and requires user approval. Without annotations, the description carries full burden and covers the most critical behavioral aspects, though it omits details about whether the original job remains unchanged.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise paragraphs: first paragraph states purpose and key benefit (saving GPU time); second paragraph has critical usage guidance. Every sentence adds value, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (billing, user approval, retry logic), the description covers the essential flow: retry mechanism, cost estimate, and required approval. No output schema is present, but the response structure (estimate inclusion) is mentioned. Lacks details on error handling or edge cases, but adequate for the main use case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds minimal new parameter-specific meaning beyond what the schema provides (e.g., job_id and github_token are already well-described in the schema). The user approval context is related but does not enhance parameter semantics directly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Retry a failed fine-tuning job from its last checkpoint,' specifying the action (retry), resource (failed fine-tuning job), and distinctive behavior (resume from checkpoint). It distinguishes itself from sibling tools like create_job or cancel_job.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs the agent to show the cost estimate to the user and get explicit approval before proceeding, with 'MUST' emphasis. Provides clear when-to-use context, though it does not explicitly mention when not to use this tool (e.g., if the job is not failed).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

show_agentA

Get details of a specific agent including capabilities, tools, and configuration.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesAgent ID

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It only states it gets details, but does not mention whether it is read-only, requires authentication, or has any side effects. For a read operation, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is front-loaded with the core action and resource, containing no unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read tool with one parameter, the description is mostly adequate. It lacks details on return format but is otherwise complete given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single parameter 'agent_id', which is described in the schema. The description adds no additional meaning beyond what the schema provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'details of a specific agent' with specific content (capabilities, tools, configuration). It distinguishes itself from sibling tool 'list_agents' by focusing on a single agent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it (to get details of a specific agent), and the sibling 'list_agents' provides an alternative for listing all agents. However, no explicit when-not or prerequisites are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

show_datasetA

Get details of a specific dataset including status, source, and metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataset_idYesDataset ID (UUID)

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral transparency. 'Get details' implies a read-only operation, but it does not explicitly state that no modifications occur, nor does it disclose permissions, rate limits, or data retention policies. The description is minimally adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence of 12 words with no filler. Every word is meaningful, making it highly scannable and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (1 parameter, no output schema, no annotations), the description is sufficient for basic understanding. However, it lacks context on response format (e.g., shape of metadata) and any caveats about data freshness or accessibility, leaving room for confusion.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter (dataset_id as UUID string). The description adds no additional context or constraints beyond what the schema already provides, so it does not improve parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description starts with 'Get details of a specific dataset', clearly stating the verb 'Get' and resource 'dataset'. It lists specific included fields (status, source, metadata) which distinguishes it from sibling tools like list_datasets (list all) or create_dataset (create new).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for retrieving details of a single dataset, but does not provide explicit guidance on when to use this tool versus alternatives (e.g., list_datasets for overview, show_evaluation for evaluation details). No exclusions or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

show_evaluationB

Get full details of a specific evaluation including status, scores, metrics, and comparison data.

ParametersJSON Schema
NameRequiredDescriptionDefault
evaluation_idYesEvaluation ID (UUID)

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden. It does not disclose that this is a read-only operation, nor any authentication requirements, rate limits, or potential side effects. The description only restates the purpose without behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one sentence of 15 words, which is efficient for a simple retrieval tool. However, it could be slightly expanded to include context like what the response contains, but it is not wasteful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (1 param, no output schema, no nested objects), the description lists some return fields (status, scores, metrics, comparison data) but does not mention error cases, access limitations, or the response format. It is adequate but incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with one parameter 'evaluation_id' described as 'Evaluation ID (UUID)'. The description does not add any additional meaning or format beyond what the schema already provides. Baseline is 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'specific evaluation', and lists what details are included: 'status, scores, metrics, and comparison data'. It distinguishes from sibling tools like 'list_evaluations' (which lists all evaluations) and 'evaluation_status' (which likely only returns status).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as 'evaluation_status' for just status, or 'list_evaluations' for an overview. No 'when-to-use' or 'when-not-to-use' information is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

show_jobA

Get full details of a specific fine-tuning job including status, base model, agent type, GPU minutes, cost, error messages, and whether it can be retried from checkpoint.

ParametersJSON Schema
NameRequiredDescriptionDefault
job_idYesJob ID (UUID)

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but the description discloses key behavioral aspects such as returning specific fields (status, base model, cost, error messages, checkpoint retryability). It could mention read-only nature explicitly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with purpose, efficiently lists details without redundancy. Every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given one parameter and no output schema, the description covers the return fields well. Could mention error conditions (e.g., job not found) but is adequate for a retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with job_id described as 'Job ID (UUID)'. The description adds no further parameter meaning beyond identifying the job, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get full details') and resource ('specific fine-tuning job'), and lists multiple fields, clearly distinguishing from siblings like 'job_status' or 'list_jobs'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies usage when full details of a single job are needed, but no explicit when-not or alternatives mentioned. Siblings like 'job_status' could serve a simpler purpose, and this is not addressed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

show_modelB

Get details of a specific trained model.

ParametersJSON Schema
NameRequiredDescriptionDefault
model_idYesModel ID (UUID)

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only says 'Get details'. It does not disclose read-only nature, required permissions, or what 'details' entails. Lacks transparency beyond basic purpose.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, zero waste, perfectly concise. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite low complexity (1 param, no annotations), description fails to specify what 'details' are returned or any usage constraints. Incomplete for an agent to use effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a well-described parameter. Description adds no additional meaning about the parameter beyond what the schema already provides. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states verb 'get' and resource 'details of a specific trained model'. It distinguishes from siblings like 'list_models' (list all) and other 'show_*' tools for different entities (agents, datasets).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives such as 'list_models' or 'show_evaluation'. Agent receives no context about appropriate usage scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_s3A

Validate S3 credentials by testing read/write access to the specified bucket. Use before submitting a job with S3 export.

ParametersJSON Schema
NameRequiredDescriptionDefault
s3_bucketYesS3 bucket name
s3_access_key_idYesAWS access key ID
s3_secret_access_keyYesAWS secret access key
s3_regionYesAWS region (e.g. us-east-1)

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description is sole source. It mentions testing read/write access, but does not disclose potential side effects like creating temporary test objects or the exact nature of the validation. Adequate but incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences. First sentence describes action, second provides usage context. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, so description should include return value hints. It does not mention what the tool returns on success/failure. Adequate for purpose and usage but missing behavioral details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all parameters with descriptions (100% coverage). The description adds no new information beyond schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the verb 'validate', resource 'S3 credentials', and context 'bucket'. It distinguishes from sibling tools which deal with jobs, evaluations, models, etc., as it's a validation tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states 'Use before submitting a job with S3 export', giving clear context. No explicit exclusions or alternatives, but the usage is straightforward for a validation tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 23 tool updatesv0.3.5
    • Addedcancel_evaluation
    • Addedcatalog_export_status
    • Addedcreate_dataset
    • Addedcreate_evaluation
    • Addeddataset_status
    • Addeddelete_dataset
    • Addedestimate_evaluation
    • Changedestimate_job1 field changed
      • addedInput schema / properties / use_case
        {
          "description": "Agent to use for the estimate (e.g. 'code_repo' for Cody, 'sera_code_repo' for SIERA). Defaults to code_repo.",
          "type": "string"
        }
    • Addedevaluation_status
    • Addedexport_catalog_model
    • Addedget_catalog_model
    • Addedget_inference_jwt
    • Addedinference_usage
    • Addedlist_agents
    • Addedlist_catalog_models
    • Addedlist_datasets
    • Addedlist_evaluations
    • Addedlist_evaluators
    • Addedlist_inference_models
    • Changedlist_supported_models1 field changed
      • addedInput schema / properties / agent
        {
          "description": "Filter models compatible with this agent (e.g. 'code_repo', 'sera_code_repo'). Omit to see all models.",
          "type": "string"
        }
    • Addedshow_agent
    • Addedshow_dataset
    • Addedshow_evaluation
  2. 17 tool updatesv1.0.0
    • First observedcancel_job
    • First observedcreate_job
    • First observeddelete_model
    • First observedestimate_job
    • First observedexport_model
    • First observedget_account
    • First observedget_balance
    • First observedimport_model
    • First observedjob_status
    • First observedlist_jobs
    • First observedlist_models
    • First observedlist_supported_models
    • First observedmodel_status
    • First observedretry_job
    • First observedshow_job
    • First observedshow_model
    • First observedvalidate_s3

TDQS

A3.7/5.0
Disambiguation5/5

Each tool targets a distinct resource and action (e.g., create_job vs. cancel_job, list_datasets vs. show_dataset). There is no overlap, and descriptions clearly differentiate related tools like estimate_job and create_job.

Naming Consistency5/5

All tools follow a consistent verb_noun pattern in snake_case (e.g., list_models, export_model, cancel_evaluation). The naming is uniform and predictable, aiding agent selection.

Tool Count4/5

38 tools is on the higher side but appropriate for a comprehensive fine-tuning platform covering jobs, evaluations, datasets, models, marketplace, inference, and account management. The scope justifies the count.

Completeness5/5

The tool surface covers the full lifecycle: job creation, estimation, cancellation, retry, and monitoring; dataset and model CRUD; evaluation workflows; account management; marketplace; and inference. No obvious gaps exist.

Maintenance

ActivityActive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • F
    license
    A
    quality
    B
    maintenance
    Provides tools for optimizing, fine-tuning, and deploying large language models with Unsloth, enabling 2x faster training with 80% less memory through model loading, fine-tuning, text generation, and model export capabilities.
    6
    7
    -
  • A
    license
    Not graded
    quality
    C
    maintenance
    Transforms AI assistants into a full ML engineering environment for training and fine-tuning models across multiple backends (local GPU, Mistral, Together AI, OpenAI) and cloud providers (Lambda Labs, RunPod, SSH-accessible VPS), with dataset management, experiment tracking, cost estimation, and deployment to Ollama/Open WebUI.
    3
    PolyForm Noncommercial 1.0.0
  • A
    license
    B
    quality
    F
    maintenance
    Exposes vLLM capabilities to AI assistants, enabling chat completions, model management, and platform-aware container control with automatic detection of Docker/Podman and GPU availability across Linux, macOS, and Windows.
    12
    8
    Apache 2.0

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/cerebrixos-org/tuning-engines-cli'

If you have feedback or need assistance with the MCP directory API, please join our Discord server