Skip to main content
Glama

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
HOSTNoHost to bind the HTTP server to (default 0.0.0.0).0.0.0.0
PORTNoPort for the streamable HTTP server (default 8000).8000
PUBLIC_URLNoPublic URL to put into the agent card.
TRACECHECK_LOG_FILENoLog file path for calls. Defaults to `/workspace/artifacts/tracecheck/calls.jsonl`, falling back to `~/.tracecheck/calls.jsonl`./workspace/artifacts/tracecheck/calls.jsonl
TRACECHECK_TRANSPORTNoServer transport, e.g. set to `http` to run as a streamable HTTP server.

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

Independent evaluation service for AI agent execution traces. Submit a goal, the step-by-step trace of what the agent did, and the claimed outcome. Receive a reliability score (0-100), detected issues, and concrete improvement suggestions. Use this to improve your own process, recover from failures, or validate long-horizon work. Free during the initial test phase.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tools
{
  "listChanged": false
}
prompts
{
  "listChanged": false
}
resources
{
  "subscribe": false,
  "listChanged": false
}

Tools

Functions exposed to the LLM to take actions

NameDescription
get_trace_schemaA

Return the exact input format TraceCheck's evaluate_trace expects, the scored dimensions and weights, and a complete example call you can copy. Call this once before your first evaluate_trace. No arguments, no side effects.

evaluate_traceA

Get an independent, deterministic reliability check of a multi-step task you (an agent) just ran. Returns a 0-100 score, the issues found, and concrete fixes to apply on your next attempt.

WHEN TO CALL: after you finish (or give up on) a multi-step task that used tools, before you report "done" to the user, or after a failure you want to learn from. Free, no API key, fast and offline (keyword heuristics, no LLM). Call get_trace_schema first if unsure of the format.

INPUT (all plain strings):

  • goal: the original task, stated specifically (what success looks like). Vague goals under 20 chars are penalised.

  • trace: your chronological steps, one per line, e.g. "1. Called search_flights(AMS->LIS) -> returned 3 results" / "2. Observed: ..." . Write what each tool RETURNED ("observed", "returned", "result:"), any error and the retry/fallback you used, and a final "Verified: ..." line.

  • outcome: the concrete final result you claim, and how it meets the goal.

  • agent_name: optional identifier for yourself (logging only).

RETURNS JSON: score (int 0-100), dimensions {goal_clarity, observability, error_recovery, goal_alignment}, issues (list), suggestions (list, max 5), summary, version.

THE LOOP: run task -> evaluate_trace -> apply the suggestions to your process -> re-run -> evaluate_trace again and compare scores/dimensions. Scores below ~60 usually mean missing observations, unrecovered errors, or no verification step.

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription

No resources

TDQS

A4.6/5.0

Scored across 2 tools

Disambiguation5/5

The two tools have clearly distinct roles: one returns format/schema information, the other performs the actual trace evaluation. The descriptions explicitly instruct when to use each, leaving no realistic ambiguity.

Naming Consistency5/5

Both names use a consistent snake_case verb_noun pattern: get_trace_schema and evaluate_trace. The convention is predictable and readable.

Tool Count3/5

Two tools is thin for most servers, but here one is a schema/introspection helper and the other is the core evaluator. It is borderline minimal rather than clearly wrong.

Completeness4/5

The surface covers the core workflow: learn the input format, then evaluate a trace. Minor gaps exist around comparing multiple evaluations or retrieving history, but agents can work around these by recording scores themselves.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues