Supports capturing streaming traces from CrewAI agents for end-to-end agentic trace evaluation across tool_use_accuracy, reasoning_quality, goal_completion, and safety_compliance.
Allows evaluating Google's AI models for safety, bias, accuracy, hallucination, reasoning, and more via batch evaluation runs.
Supports capturing streaming traces from LangChain agents for end-to-end agentic trace evaluation across tool_use_accuracy, reasoning_quality, goal_completion, and safety_compliance.
Allows evaluating OpenAI's models (e.g., GPT-4o) for safety, bias, accuracy, hallucination, reasoning, and more via batch evaluation runs.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@TrustModel MCP Server evaluate the safety and bias of my latest model"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A Model Context Protocol (MCP) server that lets any AI agent call TrustModel for trust evaluation, safety/bias analysis, and end-to-end agentic trace evaluation.
Works with Claude Code, Cursor, Windsurf, Claude Desktop, and any other MCP-compatible client.
Sign up at app.trustmodel.ai and create an API key under Settings → API Keys. Keys have the format tm-{env}-{keyid}_{secret} (e.g. tm-prod-abc12345_0123456789abcdef…).
claude mcp add trustmodel \
--env TRUSTMODEL_API_KEY=tm-prod-xxxx_yyyy \
-- npx -y @trustmodel/mcp-serverAdd to your MCP configuration file (.cursor/mcp.json or equivalent):
{
"mcpServers": {
"trustmodel": {
"command": "npx",
"args": ["-y", "@trustmodel/mcp-server"],
"env": {
"TRUSTMODEL_API_KEY": "tm-prod-xxxx_yyyy"
}
}
}
}Add to ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"trustmodel": {
"command": "npx",
"args": ["-y", "@trustmodel/mcp-server"],
"env": {
"TRUSTMODEL_API_KEY": "tm-prod-xxxx_yyyy"
}
}
}
}Related MCP server: maiat-protocol
Variable | Required | Default | Description |
| No* | — | Your TrustModel API key ( |
| No |
| Where streaming trace sessions are persisted as append-only JSONL. Sessions survive server restarts via rehydrate-on-read. |
| No |
| Tool profile. |
| No |
| Set |
| No |
| Enables the filesystem-touching Shadow Discovery tools ( |
To stay within the 5–8 tool best-practice budget (more tools degrade an agent's tool selection), the server exposes a small default set and keeps advanced tools opt-in.
Default profile (6 tools) — the daily drivers:
trustmodel_evaluate_local · trustmodel_score · trustmodel_trace_start · trustmodel_trace_step · trustmodel_trace_finalize · trustmodel_govern
Advanced — set TRUSTMODEL_PROFILE=security (or advanced / all, or TRUSTMODEL_ADVANCED_TOOLS=true) to additionally expose: trustmodel_evaluate (cloud batch), trustmodel_credits, trustmodel_upload_trace, trustmodel_evaluate_agent, trustmodel_score_agent, trustmodel_mcp_scan_server, trustmodel_shadow_discovery_*, trustmodel_redteam_*, and trustmodel_shadowai_* — 20 tools total.
claude mcp add trustmodel --env TRUSTMODEL_PROFILE=security -- npx -y @trustmodel/mcp-serverThe server exposes 18 tools across six areas. Use this table to pick the right one; full input/output docs follow below.
Tool | Group | When to use |
| Eval | Kick off a batch trust evaluation of a model (safety, bias, accuracy, …); returns an |
| Eval | Fetch status/scores for an evaluation created with |
| Eval | Check remaining API credit balance. |
| Agentic Trace | Open a streaming trace session before an agent starts working. |
| Agentic Trace | Record one reasoning step, tool call, tool result, or response as the agent runs. |
| Agentic Trace | Close the session, upload the trace, and auto-create the agent evaluation run. |
| Agentic Trace | One-shot: PUT a pre-assembled trace JSON when you didn't stream it. |
| Agentic Trace | Create an agentic evaluation run against an already-uploaded trace |
| Agentic Trace | Fetch scores/grade for an agentic evaluation run. |
| Security | Security-scan a third-party MCP server's tool list for risky/abusable tools. |
| Shadow Discovery | Scan local filesystem paths for unregistered/shadow AI usage. |
| Shadow Discovery | Detect & fingerprint OpenAI/Anthropic API keys found on disk. |
| Red Team | Launch an adversarial red-team evaluation against a model/endpoint. |
| Red Team | Fetch results for a red-team evaluation. |
| Red Team | List available red-team probes/attack categories. |
| Shadow AI | Start a Shadow AI scan to find unregistered AI use across an environment. |
| Shadow AI | Fetch results for a Shadow AI scan. |
| Shadow AI | Stream the detection events for a Shadow AI scan. |
Shadow Discovery tools (
trustmodel_shadow_discovery_*) touch the local filesystem. They are always listed, but return a skip report unlessTRUSTMODEL_AGT_DISCOVERY_ENABLED=trueis set on the server.
trustmodel_evaluateCreate a batch evaluation run against a specified AI model. The backend runs a comprehensive suite (safety, bias, accuracy, hallucination, reasoning, etc.) and returns an id you can poll with trustmodel_score.
Inputs:
model_identifier (string, required) — e.g. "gpt-4o", "claude-sonnet-4-5". Discover via GET /sdk/v1/models/.
vendor_identifier (string, required) — e.g. "openai", "anthropic", "google".
api_key (string, optional) — Vendor API key for BYOK. Omit to use TrustModel's platform key. Do not pass a TrustModel API key here — that goes in the TRUSTMODEL_API_KEY env var.
categories (string[], optional) — Category names to evaluate. Only honored when evaluation_type is "Custom" or "Score Only".
evaluation_type (string, optional, default "Custom") — One of "Custom", "Score Only", "Comprehensive", "Limited", "Quick Scan".
application_type (string, optional, default "generic") — chatbot, knowledge-agent, creation-tool, document-repository, analysis-tool, automation-agent, generic.
user_personas (string[], optional, default ["external-customer"]) — Any of external-customer, internal-employee, technical-user, domain-expert, vulnerable-groups, generic.
application_description (string, optional).
domain_expert_description (string, optional) — When user_personas includes "domain-expert". One of "cross-domain" (default), "medical", "commercial_banking".
model_config_name (string, optional) — Display name for this run.
template_id (UUID, optional), template_name (string, optional) — Reuse or rename an existing evaluation template.
trustmodel_scoreFetch the detail (status, completion %, scores) for an evaluation created via trustmodel_evaluate.
Inputs:
evaluation_id (integer or numeric string, required) — The id returned by trustmodel_evaluate.
trustmodel_creditsCheck remaining API credit balance. No inputs.
TrustModel evaluates AI agents by consuming their execution trace (thoughts, tool calls, tool results, responses) and scoring them across 4 categories: tool_use_accuracy, reasoning_quality, goal_completion, safety_compliance.
There are two ways to submit a trace — streaming (preferred for live agents) and one-shot (when you have a pre-assembled trace).
Open a session, record steps as the agent works, finalize at the end. Finalize uploads to cloud storage and auto-creates the evaluation run.
trustmodel_trace_startOpen a new trace session.
Inputs:
goal (string, required) — What the agent is trying to achieve.
name (string, required) — Display name for the evaluation run.
agent_framework (string, required) — e.g. "langchain", "crewai", "claude-code", "custom".
agent_model (string, optional) — e.g. "gpt-4o", "claude-sonnet-4-5".
user_query (string, optional) — Original user prompt, if different from goal.
expected_outcome (string, optional).
metadata (object, optional) — Free-form passthrough metadata.
Returns: { trace_id, started_at }.
trustmodel_trace_stepAppend a single step to the active session. Call once per reasoning step, tool call, tool result, or user-facing response.
Inputs:
trace_id (string, required) — From trustmodel_trace_start.
step_type (enum, required) — One of thought, think, tool_call, tool_result, observation, decision, error, human_input, response, final_answer.
content (string, required) — Human-readable text for the step. Empty string allowed.
tool_name (string, optional), tool_args (object, optional) — Use with tool_call.
tool_result (string or object, optional), tool_call_success (boolean, optional) — Use with tool_result.
model_used (string, optional), input_tokens / output_tokens (int, optional), duration_ms (int, optional), timestamp (ISO 8601, optional).
Returns: { trace_id, step_number, steps_recorded }. step_number is auto-assigned.
trustmodel_trace_finalizeClose the session, upload the trace, and auto-create the evaluation run.
Inputs:
trace_id (string, required).
final_response (string, optional), actual_outcome (string, optional), goal_achieved (boolean, optional), success (boolean, optional), total_duration_ms (int, optional — computed from step durations if omitted).
goal / name / agent_framework / agent_model / expected_outcome (all optional) — Override start-time metadata if the agent learned more at runtime.
Returns (happy path): { trace_id, file_path, expires_in, step_count, evaluation_run_id, evaluation_status, evaluation_message }.
Returns (evaluate failed after successful upload): { trace_id, file_path, expires_in, step_count, evaluation_error }. You can retry evaluation without re-uploading via trustmodel_evaluate_agent({ file_path, goal, name, agent_framework }).
trustmodel_upload_tracePUT an already-built trace JSON object to cloud storage. Returns a file_path you then pass to trustmodel_evaluate_agent.
Inputs:
trace (object, required) — Complete AgentTrace JSON.
Returns: { file_path, expires_in }.
trustmodel_evaluate_agentCreate an agentic evaluation run against a previously-uploaded trace.
Inputs:
file_path (string, required) — From trustmodel_upload_trace or trustmodel_trace_finalize.
goal / name / agent_framework (strings, required).
agent_model / expected_outcome / actual_outcome (string, optional).
goal_achieved (boolean, optional).
Returns: { evaluation_run_id, status, message }.
trustmodel_score_agentFetch the detail (scores, grade, summary) for an agentic evaluation run.
Inputs:
evaluation_run_id (integer or numeric string, required).
trustmodel_mcp_scan_serverRun AGT's MCP security scanner over a third-party MCP server's tool list to detect tool poisoning, typosquatting, hidden instructions, and rug-pull patterns. Static analysis — no LLM, no network, deterministic. Intended as a pre-registration check before an agent enables a third-party MCP server.
Inputs:
tools (array, required) — The third-party server's tool definitions (name, description, optional input schema).
Returns: A ScanReport with overall status (ok / warning / blocked), worst severity seen, and per-tool findings.
Filesystem-touching. Returns a skip report unless
TRUSTMODEL_AGT_DISCOVERY_ENABLED=true.
trustmodel_shadow_discovery_scan_pathsWalk local filesystem paths and detect agents in config files (agentmesh.yaml, crewai.yaml, mcp.json, claude_desktop_config.json, …), Dockerfiles/compose, and optionally source. Reconciles detections against a caller-supplied registry and returns the unregistered ones as shadow agents with AGT risk scoring + remediation. Static analysis only.
Inputs:
paths (string[], required) — Local paths to scan.
registry (array, optional) — Known agents (did, name, owner, …) to reconcile against.
trustmodel_shadow_discovery_fingerprint_keysFingerprint a batch of provider API keys (OpenAI / Anthropic) by calling each provider's read-only models endpoint — no inference is run. A reachable key is flagged high-risk (possible credential exposure); a revoked key is informational. Key material is never logged or returned — only a provider:****last4 fingerprint.
Inputs:
keys (string[], required) — API keys to probe.
trustmodel_redteam_evaluateRun an adversarial red-team evaluation against an OpenAI-compatible target. Probes cover 8 attack categories aligned with OWASP LLM Top 10 (2025) — prompt injection, jailbreak, PII extraction, bias elicitation, hallucination triggers, and more.
Inputs:
model_name (string, required) — e.g. "openai/gpt-oss-20b:free".
api_key (string, required), api_base_url (string, required) — e.g. "https://openrouter.ai/api/v1".
categories (string[], optional), severities (string[], optional), metadata (object, optional).
Returns: The evaluation id — poll with trustmodel_redteam_results.
trustmodel_redteam_resultsGet status, progress, overall score, per-category breakdown, and severity buckets for a red-team evaluation. Partial progress while running, full summary when completed.
Inputs:
evaluation_id (integer or numeric string, required).
trustmodel_redteam_list_probesBrowse the red-team probe library (metadata only — payloads are not exposed). Filter by category, severity, or tag.
Inputs:
category (string, optional), severity (string, optional), tag (string, optional).
trustmodel_shadowai_scanKick off a Shadow AI Discovery scan against GitHub orgs/repos and (optionally) GCP projects. Detects unregistered AI by sniffing source for LLM SDK usage (15 libraries) and listing Vertex AI endpoints, Cloud Run services with AI env vars, and BigQuery ML models.
Inputs:
github_orgs (string[], optional), github_repos (string[] of owner/name, optional), gcp_projects (string[], optional), metadata (object, optional).
Returns: The scan id — poll with trustmodel_shadowai_results, page discoveries with trustmodel_shadowai_events.
trustmodel_shadowai_resultsGet status, scan filter, total event count, and discoveries-by-system-type / by-source aggregates for a Shadow AI scan.
Inputs:
scan_id (integer or numeric string, required).
trustmodel_shadowai_eventsPage through individual discovery events — each is one discovered AI system with system_type, discovered_via, evidence, and a stable system_id fingerprint. Filter by system_type or source.
Inputs:
scan_id (integer or numeric string, required), plus optional system_type / source filters and pagination.
trustmodel_trace_start({
goal: "Book a flight from NYC to SF",
name: "Flight booking agent",
agent_framework: "claude-code",
agent_model: "claude-sonnet-4-5"
})
→ { trace_id: "trace-abc123def456", started_at: "..." }
trustmodel_trace_step({ trace_id, step_type: "thought",
content: "Need to search flights first." })
→ { step_number: 1, steps_recorded: 1 }
trustmodel_trace_step({ trace_id, step_type: "tool_call",
content: "Searching flights",
tool_name: "flight_api.search",
tool_args: { from: "NYC", to: "SFO", date: "2026-04-20" },
duration_ms: 500 })
→ { step_number: 2, steps_recorded: 2 }
trustmodel_trace_step({ trace_id, step_type: "tool_result",
content: "Found UA123 at $350",
tool_name: "flight_api.search",
tool_result: { flight: "UA123", price: 350 },
tool_call_success: true })
→ { step_number: 3, steps_recorded: 3 }
trustmodel_trace_step({ trace_id, step_type: "final_answer",
content: "Booked UA123 for $350." })
→ { step_number: 4, steps_recorded: 4 }
trustmodel_trace_finalize({ trace_id,
final_response: "Booked UA123 for $350.",
goal_achieved: true })
→ {
file_path: "agent-traces/<org>/<ts>_<uuid>.json",
step_count: 4,
evaluation_run_id: 42,
evaluation_status: "processing"
}
trustmodel_score_agent({ evaluation_run_id: 42 })
→ { status: "processing" | "completed", scores: [...], grade, overall_score, ... }Below is a real-world scenario: you ask Claude Code to perform a task while instrumenting itself with TrustModel trace capture. At the end, TrustModel scores the agent across tool-use accuracy, reasoning quality, goal completion, and safety compliance — giving you a trust report before you ship the agent to production.
Paste this prompt into Claude Code (or any MCP client with TrustModel connected):
Research the pros and cons of using WebSockets vs Server-Sent Events for
real-time notifications in a web app, while recording a TrustModel trace.
Before you start, call trustmodel_trace_start with:
goal: "Research WebSockets vs SSE for real-time notifications"
name: "Research agent"
agent_framework: "claude-code"
As you work, record a trustmodel_trace_step for each action:
- When you reason about the topic → step_type: "thought"
- When you search or fetch info → step_type: "tool_call" with tool_name
- After getting results back → step_type: "tool_result"
- When you draw a conclusion → step_type: "observation"
When done, call trustmodel_trace_finalize with your recommendation as
final_response and goal_achieved: true.
Print the evaluation_run_id so I can check the trust report.The agent researches the topic while self-tracing every reasoning step, search, and conclusion. A typical session looks like:
trustmodel_trace_start({ goal: "Research WebSockets vs SSE...", name: "Research agent", ... })
→ { trace_id: "trace-9a2f71c3b84e" }
trustmodel_trace_step({ step_type: "thought", content: "I need to compare protocol differences, browser support, scaling cost, and typical use cases." })
→ { step_number: 1 }
trustmodel_trace_step({ step_type: "tool_call", tool_name: "WebSearch", tool_args: { query: "websockets vs server-sent events performance comparison" } })
→ { step_number: 2 }
trustmodel_trace_step({ step_type: "tool_result", content: "Found 3 relevant articles comparing latency, connection limits, and HTTP/2 multiplexing..." })
→ { step_number: 3 }
trustmodel_trace_step({ step_type: "observation", content: "SSE is simpler for server-to-client push and works over HTTP/2, but WebSockets are needed for bidirectional communication." })
→ { step_number: 4 }
... (more research, comparisons, trade-off analysis) ...
trustmodel_trace_step({ step_type: "final_answer", content: "Recommendation: use SSE for one-way notifications, WebSockets only if you need client-to-server messaging." })
→ { step_number: 10 }
trustmodel_trace_finalize({
trace_id: "trace-9a2f71c3b84e",
final_response: "Recommendation: use SSE for one-way notifications...",
goal_achieved: true
})
→ {
file_path: "agent-traces/<org>/<timestamp>.json",
evaluation_run_id: 42,
evaluation_status: "processing"
}Poll trustmodel_score_agent({ evaluation_run_id: 42 }) after 1-2 minutes. TrustModel returns:
{
"status": "completed",
"overall_score": 7.6,
"grade": "C",
"scores": [
{ "category": "tool_use_accuracy", "score": 100.0 },
{ "category": "reasoning_quality", "score": 60.0 },
{ "category": "goal_completion", "score": 70.0 },
{ "category": "safety_compliance", "score": 80.0 }
],
"summary": {
"trust_dimensions": {
"safety": 9.0, "fairness": 7.0, "privacy": 10.0,
"transparency": 6.0, "robustness": 10.0, "accountability": 10.0
}
}
}A PDF/HTML report with detailed findings is also generated and accessible from the TrustModel dashboard.
Every AI agent making decisions — reviewing code, processing claims, screening candidates — needs a trust baseline before going to production. This flow gives you that baseline with zero changes to your agent's core logic: just wrap it with trace_start, record steps as it works, and trace_finalize when it's done. TrustModel handles the rest.
Active trace sessions are written as append-only JSONL at $TRUSTMODEL_TRACE_DIR/<trace_id>.jsonl (default ~/.trustmodel-mcp/traces/). Disk is the source of truth; the in-memory map is a cache that rehydrates lazily — so sessions survive an MCP server restart. On successful trustmodel_trace_finalize the local file is deleted (the trace is already in cloud storage).
Sessions idle more than 30 minutes are auto-evicted. Maximum 100 concurrent sessions per server.
Reports, evaluation history, and detailed PDF/HTML findings are available in the TrustModel dashboard at app.trustmodel.ai.
Symptom | Fix |
Server exits immediately / | Set a valid key ( |
| Set |
| Ensure Node ≥ 20.19 and run |
Tool not listed by the client | Restart the MCP client after editing its config; confirm the |
This MCP server is part of the TrustModel OSS toolkit:
CLI + SDK — trustmodel on PyPI: local trust scoring, governance, and the cloud client. pip install trustmodel.
MCP server (this repo) — @trustmodel/mcp-server on npm: exposes TrustModel to any MCP client.
MIT
Score AI output locally across the 10 TrustModel dimensions (safety, fairness, accuracy, privacy, transparency, robustness, accountability, explainability, compliance, reliability) and roll it into a 0-100 TrustScore. NO API key required — runs on this machine with a transparent heuristic judge. Returns trust_score, grade, per-dimension scores, and violations. Local scores are uncalibrated; use trustmodel_evaluate (cloud, needs a free TRUSTMODEL_API_KEY) for a calibrated, audit-ready score.
| Name | Required | Description | Default |
|---|---|---|---|
| output | Yes | The AI output text to score. | |
| context | No | Optional context the output was produced in (improves judging). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description fully discloses behavioral traits: it runs locally, requires no API key, uses a heuristic judge, and notes that local scores are uncalibrated. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise with three focused sentences. It front-loads the purpose, then adds key details and comparison. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, no output schema), the description covers everything needed: what it does, return fields, and when to use alternatives. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining that 'context' improves judging, which is not in the schema description. Params are clearly described.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it scores AI output locally across 10 dimensions and produces a TrustScore. It distinguishes itself from sibling tool 'trustmodel_evaluate' by specifying the local vs cloud nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool (local, no API key) and when to use the alternative (cloud, for calibrated scores). Provides clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check text against a governance policy pack and decide allow/block, locally. NO API key required. policy is a built-in pack id — eu-ai-act (default), nist-ai-rmf, nyc-ll144, or owasp-llm. Returns allowed/blocked, the policy id, and the list of rule violations (pattern and dimension rules). Use before letting AI output reach a user or another tool.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | The text/output to check against the policy. | |
| policy | No | Policy pack id: eu-ai-act (default), nist-ai-rmf, nyc-ll144, owasp-llm. | |
| context | No | Optional context the output was produced in. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses that the tool runs locally, returns allowed/blocked status, policy id, and rule violations. It does not explicitly state whether the tool has side effects (e.g., logging or state changes), so it misses full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loads the core purpose and unique benefit (local, no API key), and wastes no words. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters and no output schema, the description covers purpose, parameter meanings, return structure, and usage context. It is self-contained and leaves no obvious gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover 100% of parameters, baseline is 3. The tool description reinforces the policy pack list and clarifies the 'context' parameter, but adds limited new semantic information beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Check text'), resource ('against a governance policy pack'), and result ('decide allow/block'). It also differentiates from siblings by emphasizing 'locally' and 'NO API key required'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use: 'Use before letting AI output reach a user or another tool.' It also notes 'NO API key required' as a key trait. However, it does not explicitly mention when not to use or compare to sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the current trust score / detail for a previous LLM evaluation created via trustmodel_evaluate. Takes an integer evaluation_id and returns the evaluation detail including scores.
| Name | Required | Description | Default |
|---|---|---|---|
| evaluation_id | Yes | Integer evaluation ID returned from a previous trustmodel_evaluate call. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It describes a read operation ('Get') with no side effects mentioned, which is appropriate. However, it lacks details on idempotency, rate limits, or potential errors. The description is clear but could be more transparent about behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no redundancy: first states purpose, second covers input and output. Every word contributes value, and it is front-loaded. Very concise without being terse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description is nearly complete. It explains purpose, input, and general output. It could mention possible error conditions or return format more precisely, but given the simplicity, it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter. The tool description adds context that the evaluation_id comes from a previous trustmodel_evaluate call, but this is already in the parameter's schema description. Thus, the description adds minimal extra meaning beyond the schema, fitting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a trust score/detail for a previous evaluation, with specific verb 'Get' and resource 'trust score/detail'. It distinguishes from sibling tools like trustmodel_trace_* and trustmodel_evaluate_local by focusing on retrieval of past evaluation results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use after calling trustmodel_evaluate by specifying it is for a previous evaluation created via that tool, but it does not explicitly state when to use this over alternatives nor provide exclusions. Guidance is adequate but minimal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Close an active trace session, serialize the captured steps, upload to TrustModel cloud storage, and auto-create an agentic evaluation run. Returns both a file_path (for record-keeping / retry) and an evaluation_run_id that can be polled with trustmodel_score_agent.
| Name | Required | Description | Default |
|---|---|---|---|
| goal | No | Override `goal` if the agent learned more at runtime. | |
| name | No | Override `name` provided at trace_start. | |
| success | No | Whether the run is considered successful overall. | |
| trace_id | Yes | Trace handle returned by trustmodel_trace_start. | |
| agent_model | No | Override `agent_model` provided at trace_start. | |
| goal_achieved | No | Whether the agent achieved its goal. | |
| actual_outcome | No | Optional description of what actually happened. | |
| final_response | No | The final user-facing answer from the agent. | |
| agent_framework | No | Override `agent_framework` provided at trace_start. | |
| expected_outcome | No | Override `expected_outcome` provided at trace_start. | |
| total_duration_ms | No | Total execution time in ms. If omitted, computed from step durations. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states that the active trace session is closed and an evaluation run is auto-created, implying irreversibility. However, it does not disclose side effects like whether data can be overwritten, any rate limits, or whether finalization can be repeated. Since no annotations are provided, the description should be more thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the core action and lists subsequent steps efficiently. No wasted words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (11 parameters, no output schema, no annotations), the description covers the main workflow and return values. It mentions how to use evaluation_run_id. However, it could be more complete by explicitly stating prerequisites (e.g., trace must be started) and explaining what 'agentic evaluation run' means. The schema handles parameter details well.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so each parameter is already explained. The tool description does not add further meaning to the parameters but provides context for return values (file_path and evaluation_run_id). Per guidelines, baseline is 3 when schema coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verbs (close, serialize, upload, auto-create) and clearly identifies the resource (active trace session). It distinguishes itself from sibling tools like trustmodel_trace_start (opens session) and trustmodel_trace_step (logs steps), making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used after starting a trace (since it closes an active session) and mentions polling with trustmodel_score_agent, but it does not provide explicit guidance on when to choose this tool over alternatives like trustmodel_evaluate_local or trustmodel_govern. Lacks explicit usage conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Open a new agent-trace capture session. Returns a trace_id that must be passed to every trustmodel_trace_step and the final trustmodel_trace_finalize. Capture steps (thoughts, tool calls, tool results, responses) as your agent executes; finalize when done to upload + auto-create an evaluation run.
| Name | Required | Description | Default |
|---|---|---|---|
| goal | Yes | What the agent is trying to achieve. Required; feeds the evaluation run. | |
| name | Yes | Display name for the evaluation run (shown in TrustModel UI). | |
| metadata | No | Free-form passthrough metadata stored inside the trace file. | |
| user_query | No | Original user prompt that triggered the run, if different from `goal`. | |
| agent_model | No | Underlying LLM the agent is using (e.g. 'gpt-4o', 'claude-sonnet-4-5'). | |
| agent_framework | Yes | Framework the agent is built with (e.g. 'langchain', 'crewai', 'claude-code', 'custom'). | |
| expected_outcome | No | Optional description of the expected outcome. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that the tool returns a trace_id and outlines the capture session lifecycle, though it does not mention rate limits, errors, or auth needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words, front-loaded with the primary action and outcome.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the overall flow and return value, but without an output schema, some details about the trace_id are implied rather than explicit. Given the complexity, it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3 is appropriate. The description does not add meaning beyond the schema's detailed parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool opens a new agent-trace capture session and returns a trace_id, distinguishing it from sibling tools like trustmodel_trace_step and trustmodel_trace_finalize.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the required sequence: use this tool to start, pass the trace_id to step and finalize, and finalize to upload. This provides clear context for when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Record a single step in an active trace session. Call once per reasoning step, tool call, tool result, or user-facing response. Step numbers are auto-assigned (1-based). Requires a trace_id from trustmodel_trace_start.
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | Human-readable text for this step. Empty string allowed. | |
| trace_id | Yes | Trace handle returned by trustmodel_trace_start. | |
| step_type | Yes | Kind of step: 'thought'/'think' (reasoning), 'tool_call', 'tool_result', 'observation', 'decision', 'error', 'human_input', 'response'/'final_answer' (user-facing output). | |
| timestamp | No | ISO 8601 timestamp. Defaults to the current time if omitted. | |
| tool_args | No | Arguments passed to the tool (use with step_type='tool_call'). | |
| tool_name | No | Name of the tool invoked (use with step_type='tool_call' or 'tool_result'). | |
| model_used | No | Model used for this step's reasoning. | |
| duration_ms | No | How long this step took, in milliseconds. | |
| tool_result | No | Result returned by the tool (use with step_type='tool_result'). | |
| input_tokens | No | Input token count for this step. | |
| output_tokens | No | Output token count for this step. | |
| tool_call_success | No | Whether the tool call succeeded (use with step_type='tool_result'). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses auto-assigned step numbers (1-based) and default timestamp. No annotations provided, so description carries burden; however, it does not detail error handling, side effects, or behavior for invalid trace_id.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, front-loaded with purpose. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main usage and required parameter, but does not explain return value or error scenarios. Given high parameter count (12) and no output schema, additional context about success/failure would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters have descriptions. The description adds minimal extra meaning beyond the schema (e.g., step number auto-assignment). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Record a single step' and resource 'active trace session'. Distinguishes from siblings (trace_start, trace_finalize) by specifying it is for step recording during an active session.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly describes when to call ('once per reasoning step, tool call, tool result, or user-facing response') and prerequisite ('Requires a trace_id from trustmodel_trace_start'). Lacks explicit when-not-to-use or alternatives, but context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.2.0trustmodel_evaluate_localtrustmodel_governtrustmodel_scoretrustmodel_trace_finalizetrustmodel_trace_starttrustmodel_trace_stepScored across 6 tools
Each tool serves a clear, distinct purpose: trace capture (start/step/finalize), local evaluation, governance check, and retrieval of cloud evaluation results. There is no overlapping functionality; a tool for one task cannot be easily confused with another.
All tools follow a consistent 'trustmodel_verb[_modifier]' pattern using snake_case. Verbs like score, trace_start, trace_step, trace_finalize, evaluate_local, and govern are descriptive and follow a predictable convention.
With 6 tools covering trace capture (3), evaluation (2 including retrieval), and governance (1), the count is well-scoped for an agentic evaluation and governance server. No tool is extraneous, and the set is not too sparse or bloated.
The set has notable gaps: trustmodel_trace_finalize references trustmodel_score_agent which does not exist, and trustmodel_score expects an evaluation_id from a missing trustmodel_evaluate tool. The cloud evaluation path is incomplete, leaving agents unable to retrieve scores from trace evaluations or perform cloud evaluations.
The trust harness for AI agents. Set what an agent can do before it acts.
Trust signals for AI agents: an open agent-readiness standard and developer tool guide. Read-only.
Zero-trust gateway for AI agents: score tool calls, verify agent cards, enforce policy, audit.
Deterministic runtime safety for AI agents: scan PII, gate tool actions, verify LLM output.
