Skip to main content
Glama

tracehub-mcp

CI Python 3.11+ License

Give your AI assistant a direct line into your observability backend. tracehub-mcp is an MCP (Model Context Protocol) server that lets Claude, Cursor, Windsurf, Gemini CLI, or any MCP client query OpenTelemetry traces from your LLM/GenAI application and reason about them — find expensive calls, debug errors, compare model performance, track token usage — without you copy-pasting trace JSON into a chat window.

It speaks OpenTelemetry's gen_ai.* semantic conventions natively, so it understands prompts, completions, token usage, and finish reasons as first-class concepts, not just generic span attributes.

tracehub-mcp started as a fork of traceloop/opentelemetry-mcp-server (Apache 2.0) — full attribution and fork history are in NOTICE. It's grown into a 5-backend, security-hardened server maintained independently under mcpsmiths; see What's Different From Upstream below for the parts that are new here.


Table of Contents


Related MCP server: otel-instrumentation-mcp

Quick Start

tracehub-mcp is not yet published to PyPI (v0.1, pre-release — Trusted Publisher setup is a separate pending step). Until then, run it straight from GitHub with uv — no clone required:

// claude_desktop_config.json
{
  "mcpServers": {
    "tracehub-mcp": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/mcpsmiths/tracehub-mcp.git", "tracehub-mcp"],
      "env": {
        "BACKEND_TYPE": "jaeger",
        "BACKEND_URL": "http://localhost:16686"
      }
    }
  }
}

That's it. Ask your assistant: "Show me traces with errors from the last hour."

Once this is live on PyPI, the same config collapses to "command": "uvx", "args": ["tracehub-mcp"] — see Installation for the from-source path used by everything below in the meantime.


Supported Backends

  • Jaeger — local/self-hosted, the most common open-source trace backend. No auth required.

  • Grafana Tempo — local or Grafana Cloud, TraceQL-native search.

  • Traceloop — cloud LLM observability platform, API-key auth.

  • Datadog — cloud APM, requires an API key and an Application key.

  • Sentry — cloud or self-hosted, requires an auth token and an organization slug.

All five implement the same BaseBackend interface, so every MCP tool works identically regardless of which one you point the server at. See Configuration for per-backend setup.


What's Different From Upstream

Upstream opentelemetry-mcp-server shipped Jaeger, Tempo, and Traceloop. tracehub-mcp adds Datadog and Sentry as full backends — not thin wrappers, but complete implementations of every tool (search, span search, trace hydration, service discovery, health checks). Along the way, all five backends — including the three inherited from upstream — were hardened to a consistent bar:

  • HTTPS-only enforcement on cloud backends. Datadog and Sentry both refuse to start against a plain http:// URL, because their auth is a bearer token / API+App key pair that has no business going out over plaintext.

  • Query-injection-safe escaping. Every value spliced into a Datadog span-search query or a Sentry Discover query is escaped and exact-quoted; field names (which are less obviously untrusted, since they come from the MCP tool's filters parameter) are validated against an allowlist pattern before being spliced into the query string, closing off structural injection through a crafted field name.

  • Bounded pagination on every backend that paginates via cursor (Datadog, Sentry) — a search stops at the requested limit or when the backend stops returning a continuation cursor, whichever comes first, so a single tool call can't degrade into an unbounded crawl.

  • Exact-ID re-verification. Where a backend's search API can return neighbors instead of an exact match (notably Datadog's trace reconstruction from grouped spans), every result is re-checked against the exact ID that was asked for before being returned.

  • No fabricated data on malformed responses, in the backends we built. Datadog and Sentry reject a span outright — rather than substituting a placeholder like now() for a missing timestamp or a literal "unknown" for a missing service_name/operation_name — since a fabricated value would silently corrupt trace ordering, duration aggregation, and any tool that groups by service or operation. (The three backends inherited from upstream — Jaeger, Tempo, Traceloop — predate this discipline and haven't been retrofitted; that's deliberate scope discipline, not an oversight, mirroring this project's own precedent of not reaching into shared/inherited code without full regression coverage for it.)

All of this is backed by 213 passing tests (2 skipped, zero regressions), a clean ruff check and mypy --strict run, and two rounds of adversarial CodeRabbit review on the new backends.


Installation

Until tracehub-mcp lands on PyPI, there are two supported ways to run it — both work with any MCP client, and both are used throughout this README.

Option 1: Run directly from GitHub (no clone)

uvx --from git+https://github.com/mcpsmiths/tracehub-mcp.git tracehub-mcp --backend jaeger --url http://localhost:16686

This is what the Quick Start config above uses. uv fetches the repo, builds an isolated environment, and runs the tracehub-mcp entry point — same experience as uvx tracehub-mcp will be once the package is published.

Option 2: Clone and run from source

git clone https://github.com/mcpsmiths/tracehub-mcp.git
cd tracehub-mcp
uv sync

uv run tracehub-mcp --backend jaeger --url http://localhost:16686

Use this if you're developing locally, want to pin to a specific commit, or want the dev tooling installed (uv sync --group dev).

Prerequisites: Python 3.11+ and uv. pipx/pip also work once the package is on PyPI (pipx install tracehub-mcp, pip install tracehub-mcp) — not yet, today.


Configuration

Configuration comes from environment variables, CLI flags, or both. Precedence: CLI arguments > environment variables > defaults.

# .env (see .env.example)
BACKEND_TYPE=jaeger
BACKEND_URL=http://localhost:16686
# Equivalent via CLI flags
tracehub-mcp --backend jaeger --url http://localhost:16686

All Configuration Options

Variable

Type

Default

Description

BACKEND_TYPE

string

jaeger

Backend type: jaeger, tempo, traceloop, datadog, or sentry

BACKEND_URL

URL

-

Backend API endpoint (required)

BACKEND_API_KEY

string

-

API key/auth token (required for Traceloop, Datadog, and Sentry)

BACKEND_APP_KEY

string

-

Application key (Datadog only, in addition to BACKEND_API_KEY)

BACKEND_SENTRY_ORG

string

-

Organization slug (required for Sentry)

BACKEND_SENTRY_PROJECT

string

-

Project slug (optional for Sentry, narrows queries to one project)

BACKEND_ENVIRONMENTS

string

prd

Comma-separated environments (Traceloop only)

BACKEND_TIMEOUT

float

30

Request timeout in seconds

LOG_LEVEL

string

INFO

Logging level: DEBUG, INFO, WARNING, ERROR

MAX_TRACES_PER_QUERY

integer

500

Parsed and validated (1-1000) but not currently wired into any query — each tool's own limit parameter is the real per-call cap

Every CLI flag has a matching env var (--backend/BACKEND_TYPE, --url/BACKEND_URL, --api-key/BACKEND_API_KEY, --app-key/BACKEND_APP_KEY, --sentry-org/BACKEND_SENTRY_ORG, --sentry-project/BACKEND_SENTRY_PROJECT, --environments/BACKEND_ENVIRONMENTS). Run tracehub-mcp --help for the full list.

Backend-Specific Setup

BACKEND_TYPE=jaeger
BACKEND_URL=http://localhost:16686

No API key required. search_traces and search_spans_tool both require a service_name parameter — Jaeger's API is optimized for per-service queries, so querying across all services isn't supported. Discover service names first with list_services.

BACKEND_TYPE=tempo
BACKEND_URL=http://localhost:3200

No API key required for a local/self-hosted install. Search uses TraceQL under the hood; service_name is optional.

BACKEND_TYPE=traceloop
BACKEND_URL=https://api.traceloop.com
BACKEND_API_KEY=your_api_key_here

The API key encodes project information — the backend always uses a project slug of "default", and Traceloop resolves the actual project/environment from the key itself.

BACKEND_TYPE=datadog
# US site (default): https://api.datadoghq.com
# EU site:            https://api.datadoghq.eu
BACKEND_URL=https://api.datadoghq.com
BACKEND_API_KEY=your_api_key_here
BACKEND_APP_KEY=your_application_key_here

Datadog requires both an API key and an Application key — a single key is not enough for span/trace queries, even though ingestion only needs the API key. Trace search uses Datadog's span search query syntax rather than TraceQL or Jaeger-style tag params, and traces are reconstructed from grouped spans since Datadog has no trace-level lookup endpoint. The backend also refuses a plain http:// URL — see What's Different From Upstream.

Troubleshooting: a 403 from the Datadog API almost always means the Application key (not the API key) is missing or invalid. If you're on the EU site, double check BACKEND_URL is https://api.datadoghq.eu, not the US default.

BACKEND_TYPE=sentry
# SaaS (may be region-specific, e.g. https://us.sentry.io):
BACKEND_URL=https://sentry.io
BACKEND_API_KEY=your_auth_token_here
BACKEND_SENTRY_ORG=your-org-slug
# Optional: narrow queries to one project
BACKEND_SENTRY_PROJECT=your-project-slug

Sentry requires both an auth token and an organization slug — every endpoint this backend calls is organization-scoped. Trace search uses Sentry's search syntax against the Discover/Explore Events API. Unlike Datadog, Sentry does have a native trace-lookup endpoint, so get_trace calls it directly instead of reconstructing a trace from spans — search_traces still discovers candidate trace IDs via a span search first, since Sentry's search surface is itself span-centric. Like Datadog, this backend refuses a plain http:// URL.

Troubleshooting: a 403/401 from the Sentry API almost always means the auth token is missing, invalid, or lacks the necessary scopes. A 404 on an org-scoped endpoint usually means the organization slug is wrong. For a self-hosted install, BACKEND_URL should be the install's own base URL, not https://sentry.io. Some of the tracing endpoints this backend depends on are newer/experimental on Sentry's side and may not be available on every plan or self-hosted version — see the module docstring in backends/sentry.py for specifics.

Transport Modes

# stdio (default) — local use, Claude Desktop, single process
tracehub-mcp                      # pipx/pip install
uv run tracehub-mcp               # from-source install

# HTTP — remote access, multiple clients, network deployment, sample applications
tracehub-mcp --transport http --host 0.0.0.0 --port 8000              # pipx/pip install
uv run tracehub-mcp --transport http --host 0.0.0.0 --port 8000       # from-source install

With HTTP transport, clients connect to http://<host>:<port>/mcp (streamable-HTTP, for compatibility across MCP clients).


MCP Client Setup

Every example below uses the from-source install (Option 2 above); swap in the uvx --from git+... form from Quick Start if you'd rather skip cloning.

Config file location:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

  • Windows: %APPDATA%\Claude\claude_desktop_config.json

Jaeger (no auth):

{
  "mcpServers": {
    "tracehub-mcp": {
      "command": "uv",
      "args": ["--directory", "/absolute/path/to/tracehub-mcp", "run", "tracehub-mcp"],
      "env": {
        "BACKEND_TYPE": "jaeger",
        "BACKEND_URL": "http://localhost:16686"
      }
    }
  }
}

Datadog (API key + App key):

{
  "mcpServers": {
    "tracehub-mcp": {
      "command": "uv",
      "args": ["--directory", "/absolute/path/to/tracehub-mcp", "run", "tracehub-mcp"],
      "env": {
        "BACKEND_TYPE": "datadog",
        "BACKEND_URL": "https://api.datadoghq.com",
        "BACKEND_API_KEY": "your_api_key_here",
        "BACKEND_APP_KEY": "your_application_key_here"
      }
    }
  }
}

Sentry (auth token + org slug):

{
  "mcpServers": {
    "tracehub-mcp": {
      "command": "uv",
      "args": ["--directory", "/absolute/path/to/tracehub-mcp", "run", "tracehub-mcp"],
      "env": {
        "BACKEND_TYPE": "sentry",
        "BACKEND_URL": "https://sentry.io",
        "BACKEND_API_KEY": "your_auth_token_here",
        "BACKEND_SENTRY_ORG": "your-org-slug"
      }
    }
  }
}

Or use the bundled wrapper script for easy backend switching during local dev:

{
  "mcpServers": {
    "tracehub-mcp": {
      "command": "/absolute/path/to/tracehub-mcp/start_locally.sh"
    }
  }
}

(the script ships Jaeger/Traceloop/Tempo blocks only, with Jaeger active by default — to switch, comment out the active block and uncomment the one you want; Datadog/Sentry aren't in the script, so add their export lines manually).

Claude Code reads the same MCP server config as Claude Desktop. Once configured:

claude mcp list
claude "Show me traces with errors from the last hour"

Settings → MCP → Add new MCP Server, then use the same JSON shape as Claude Desktop above (Cursor's config omits the outer mcpServers wrapper in some versions — check your Cursor version's MCP settings UI for the exact shape it expects).

Settings → MCP Servers → Add New MCP Server, then use the same JSON shape as Claude Desktop above.

Config file: ~/.gemini/config.json, same JSON shape as Claude Desktop above. Then:

gemini "Analyze token usage for gpt-4 requests today"

Tools Reference

tracehub-mcp exposes 11 MCP tools:

Tool

Description

Use Case

search_traces

Search traces with simple params or advanced filters

Find specific requests or patterns

search_spans_tool

Search individual spans (not grouped into traces)

Find LLM tool calls, specific ops

get_trace

Get complete trace details by trace ID

Deep-dive into a single trace

get_llm_usage

Aggregate token usage metrics

Track costs and usage trends

list_services

List available services

Discover what's instrumented

find_errors

Find traces with errors

Debug failures quickly

list_llm_models

Discover models in use, with usage stats

Track model adoption, shadow AI

get_llm_model_stats

Latency/token percentiles + finish reasons for one model

Compare model efficiency

get_llm_expensive_traces

Find highest token-usage traces

Cost optimization

get_llm_slow_traces

Find slowest traces by duration

Latency debugging

list_llm_tools_tool

Discover LLM tool/function calls (traceloop.span.kind == tool)

Track agent tool usage

Backend Support Matrix

Feature

Jaeger

Tempo

Traceloop

Datadog

Sentry

Search traces

✓†

✓‡

Search spans

*

Get trace by ID

✓†

Advanced filters

Error traces

All LLM tools

* Jaeger requires the service_name parameter for span search. † Datadog has no trace-level API; traces are reconstructed by searching spans and grouping by trace_id, with every result re-verified against the exact ID requested. ‡ Sentry does have a native trace-lookup endpoint (unlike Datadog), so get_trace calls it directly; search_traces still discovers candidate trace IDs via a span search first, since Sentry's search surface is itself span-centric.

Key Tool Details

search_traces

{
  "service_name": "my-app",
  "start_time": "2024-01-01T00:00:00Z",
  "end_time": "2024-01-01T23:59:59Z",
  "gen_ai_system": "openai",
  "gen_ai_request_model": "gpt-4",
  "min_duration_ms": 1000,
  "has_error": false,
  "limit": 50
}

Parameters: service_name, operation_name, start_time/end_time (ISO 8601), min_duration_ms/max_duration_ms, gen_ai_system, gen_ai_request_model, gen_ai_response_model, has_error, tags, filters (see Generic Filter System), limit (1-1000, default 100). Returns trace summaries with token counts.

get_trace

{ "trace_id": "abc123def456" }

Returns the full trace tree: all spans with attributes, parsed OpenTelemetry gen_ai.* data for LLM spans, per-span token usage, and error information.

get_llm_usage

{
  "start_time": "2024-01-01T00:00:00Z",
  "end_time": "2024-01-01T23:59:59Z",
  "service_name": "my-app",
  "gen_ai_system": "openai",
  "limit": 1000
}

Returns aggregated prompt/completion/total tokens, broken down by model and by service, plus request counts.

list_services — no parameters. Returns the list of instrumented service names.

find_errors

{
  "start_time": "2024-01-15T14:00:00Z",
  "service_name": "my-app",
  "limit": 50
}

Returns error messages, error types, truncated stack traces, and LLM-specific error info.

list_llm_models / get_llm_model_stats / get_llm_expensive_traces / get_llm_slow_traces / list_llm_tools_tool / search_spans_tool are documented in detail, with worked examples, in CLAUDE.md — this README covers the shape every tool shares; CLAUDE.md is the fuller reference for exact parameters and response fields on the LLM-analysis tools.


Generic Filter System

search_traces and search_spans_tool both accept a filters list in addition to (or instead of) their simple named parameters, for advanced queries. Each filter is:

{
  "field": "gen_ai.usage.total_tokens",
  "operator": "gt",
  "value": 5000,
  "value_type": "number"
}
  • field — dotted attribute name, e.g. gen_ai.usage.prompt_tokens, traceloop.span.kind, service.name

  • operator — see table below

  • value — single value (most operators) or values — list (for in, not_in, between)

  • value_type"string", "number", or "boolean"

Category

Operators

String

equals, not_equals, contains, not_contains, starts_with, ends_with, in, not_in

Number

equals, not_equals, gt, lt, gte, lte, between, in, not_in

Boolean

equals, not_equals

Existence

exists, not_exists (no value needed)

Multiple filters combine with AND logic. Legacy simple parameters (service_name, gen_ai_request_model, etc.) still work and are converted to filters internally — mix and match freely.

The server uses a hybrid filtering strategy: filters are pushed to the backend's native query language when supported (TraceQL for Tempo, span-search syntax for Datadog, Discover syntax for Sentry), and applied client-side afterward for anything the backend can't express natively.

Backend

Native filter support

Notes

Tempo (TraceQL)

equals, not_equals, gt, lt, gte, lte, contains (regex), in (OR), exists, not_exists

Traceloop

equals, not_equals, gt, lt, gte, lte

Datadog

Most operators via span-search syntax

Field names validated against an allowlist before being spliced into the query

Sentry

Most operators via Discover search syntax

Same field-name allowlisting as Datadog

Jaeger

equals (via tags only)

Requires service_name

Example — expensive OpenAI traces:

{
  "filters": [
    { "field": "gen_ai.system", "operator": "equals", "value": "openai", "value_type": "string" },
    { "field": "gen_ai.usage.total_tokens", "operator": "gt", "value": 5000, "value_type": "number" }
  ]
}

For the full semantic-convention attribute list (gen_ai.* vs legacy llm.*, token-naming variants across providers, finish-reason values, and the token-calculation fallback chain), see CLAUDE.md.


Example Queries

Find Expensive OpenAI Operations

Ask: "Show me OpenAI traces from the last hour that took longer than 5 seconds"

Tool call: search_traces

{
  "service_name": "my-app",
  "gen_ai_system": "openai",
  "min_duration_ms": 5000,
  "start_time": "2024-01-15T10:00:00Z",
  "limit": 20
}

Response:

{
  "traces": [
    {
      "trace_id": "abc123...",
      "service_name": "my-app",
      "operation_name": "chat.completions",
      "status": "OK",
      "duration_ms": 8250,
      "span_count": 3,
      "llm_span_count": 1,
      "total_tokens": 4523,
      "has_errors": false
    }
  ],
  "count": 1
}

Analyze Token Usage by Model

Ask: "How many tokens did we use for each model today?"

Tool call: get_llm_usage

{
  "start_time": "2024-01-15T00:00:00Z",
  "end_time": "2024-01-15T23:59:59Z",
  "service_name": "my-app"
}

Response:

{
  "period": { "start_time": "2024-01-15T00:00:00Z", "end_time": "2024-01-15T23:59:59Z" },
  "filters": { "service_name": "my-app" },
  "summary": {
    "total_requests": 487,
    "total_prompt_tokens": 82140,
    "total_completion_tokens": 43290,
    "total_tokens": 125430
  },
  "by_model": {
    "gpt-4": { "requests": 156, "prompt_tokens": 58300, "completion_tokens": 26900, "total_tokens": 85200 },
    "gpt-3.5-turbo": { "requests": 331, "prompt_tokens": 23840, "completion_tokens": 16390, "total_tokens": 40230 }
  },
  "by_service": {
    "my-app": { "requests": 487, "prompt_tokens": 82140, "completion_tokens": 43290, "total_tokens": 125430 }
  }
}

Find Traces with Errors

Ask: "Show me all errors from the last hour"

Tool call: find_errors

{
  "start_time": "2024-01-15T14:00:00Z",
  "service_name": "my-app",
  "limit": 10
}

Response:

{
  "count": 1,
  "error_traces": [
    {
      "trace_id": "def456...",
      "service_name": "my-app",
      "operation_name": "chat.completions",
      "start_time": "2024-01-15T14:23:15Z",
      "duration_ms": 1200,
      "status": "ERROR",
      "span_count": 2,
      "llm_span_count": 1,
      "total_tokens": 310,
      "has_errors": true,
      "error_spans": [
        {
          "span_id": "span789...",
          "operation_name": "chat.completions",
          "service_name": "my-app",
          "status": "ERROR",
          "error_message": "RateLimitError: Too many requests",
          "error_type": "openai.error.RateLimitError",
          "is_llm_error": true,
          "llm_provider": "openai",
          "llm_model": "gpt-4"
        }
      ]
    }
  ]
}

Compare Model Performance

Ask: "What's the performance difference between GPT-4 and Claude?"

Tool call 1: get_llm_model_stats for gpt-4

{ "model_name": "gpt-4", "start_time": "2024-01-15T00:00:00Z" }

Tool call 2: get_llm_model_stats for claude-3-opus

{ "model_name": "claude-3-opus-20240229", "start_time": "2024-01-15T00:00:00Z" }

Investigate High Token Usage

Ask: "Which requests used the most tokens today?"

Tool call: get_llm_expensive_traces

{ "limit": 10, "start_time": "2024-01-15T00:00:00Z", "min_tokens": 5000 }

Common Workflows

Cost Optimization

  1. get_llm_expensive_traces — find the highest-token requests

  2. get_llm_usage — see which models are costing the most

  3. get_trace on a specific trace_id — inspect the exact prompt/response

Performance Debugging

  1. get_llm_slow_traces — identify latency outliers

  2. find_errors — check for failure patterns

  3. get_llm_model_stats — check finish-reason distribution for truncation

Model Adoption Tracking

  1. list_llm_models — see every model actually being called

  2. get_llm_model_stats per model — compare performance

  3. Scan list_llm_models results for unexpected models/services (shadow AI)


Development

git clone https://github.com/mcpsmiths/tracehub-mcp.git
cd tracehub-mcp
uv sync --group dev   # pulls in pytest, mypy, ruff, etc. for local iteration

# Tests (213 passed, 2 skipped at time of writing)
uv run pytest

# With coverage
uv run pytest --cov=opentelemetry_mcp --cov-report=html

# Format, lint, type-check
uv run ruff format .
uv run ruff check .
uv run mypy src/

CI (.github/workflows/ci.yml) runs Ruff and mypy (strict) on every push, plus the full pytest suite.


Troubleshooting

Backend connection issues:

curl http://localhost:16686/api/services   # Jaeger
curl http://localhost:3200/api/search/tags  # Tempo

Authentication errors: confirm your key is set —

export BACKEND_API_KEY=your_key_here
# or: tracehub-mcp --api-key your_key_here

No traces found:

  • Check the time range (use recent timestamps)

  • Verify service names with list_services

  • Try searching without filters first

Token usage shows zero:

  • Confirm your traces have OpenTelemetry gen_ai.* (or legacy llm.*) instrumentation

  • Inspect raw span attributes with get_trace

Datadog/Sentry-specific issues: see the troubleshooting notes under each backend in Configuration.


Roadmap

Two ideas are deliberately not built yet — they're being deferred until v0.1 ships and gets real usage feedback, rather than guessed at up front:

  • Cross-backend correlation — querying multiple configured backends in a single call and correlating results across them (e.g. a Datadog trace and its downstream Sentry error, joined).

  • Agent-native triage — tools that flag a likely root cause rather than just returning raw trace data, so an agent can act on a diagnosis instead of re-deriving one from a trace dump every time.

Beyond that, the next backends under research (in order, not yet started): Grafana Cloud, New Relic, Honeycomb, AWS X-Ray.

Carried over from upstream's older roadmap and still pending, re-prioritized behind the above rather than dropped: cost calculation with built-in pricing tables, model performance comparison tools, prompt pattern analysis, MCP resources for common queries, a caching layer for frequent queries, and SigNoz/ClickHouse backend support.

None of the above is shipped. Everything documented elsewhere in this README is.


Contributing

Contributions are welcome. Before opening a PR, make sure:

  1. All tests pass: uv run pytest

  2. Code is formatted: uv run ruff format .

  3. No linting errors: uv run ruff check .

  4. Type checking passes: uv run mypy src/

License

Apache License 2.0 — see LICENSE. This project is a fork of traceloop/opentelemetry-mcp-server; full attribution and the fork relationship are documented in NOTICE.

Support

Available Tools

11 tools
find_errorsB

Find traces with errors.

Including detailed error messages, stack traces, and LLM-specific error information.

Args: start_time: Start time in ISO 8601 format end_time: End time in ISO 8601 format service_name: Filter by service name limit: Maximum error traces to return (default: 100)

Returns: JSON string with error traces

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
end_timeNo
start_timeNo
service_nameNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It discloses that results include detailed error messages, stack traces, and LLM-specific error information, which adds some transparency about output content. However, it does not state whether the operation is read-only, what permissions are required, or how pagination/limits behave beyond the limit parameter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the purpose and structured with Args/Returns sections, making it easy to scan. It is appropriately sized. The Returns section is somewhat redundant because an output schema exists, but it does not detract much from overall conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity, four parameters, and an output schema, the description is adequate but has gaps. It covers purpose, parameters, and return format. However, it lacks usage guidance relative to sibling tools and does not disclose behavioral traits such as read-only safety, which is more important because no annotations are present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate; it does so for all four parameters. It gives time format (ISO 8601), explains service_name as a filter, and defines limit as the maximum number of error traces with a default of 100. It could add more detail (e.g., timezone, inclusive/exclusive bounds), but it substantially covers the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb and resource: 'Find traces with errors.' This distinguishes it from generic trace searching siblings like search_traces. However, it does not explicitly differentiate itself from search_traces or get_trace, so an agent must infer the distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as search_traces or get_trace. It does not mention prerequisites, exclusions, or contextual conditions for selection. Usage is only implied by the tool's purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_llm_expensive_tracesA

Find traces with highest LLM token usage.

Useful for cost optimization and identifying inefficient prompts.

Args: limit: Maximum number of traces to return (default: 10) start_time: Start time in ISO 8601 format (e.g., 2024-01-01T00:00:00Z) end_time: End time in ISO 8601 format min_tokens: Minimum token count threshold (only return traces above this) service_name: Filter by service name gen_ai_request_model: Filter by requested model name (e.g., "gpt-4") gen_ai_response_model: Filter by actual model used (e.g., "gpt-4-0613")

Returns: JSON string with top N most expensive traces sorted by total token usage

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
end_timeNo
min_tokensNo
start_timeNo
service_nameNo
gen_ai_request_modelNo
gen_ai_response_modelNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It discloses the sort order (top N by total token usage) and threshold semantics, but says nothing about permissions, pagination, rate limits, or whether only LLM-instrumented traces are considered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the purpose, then args, then returns, with no redundancy. 'Useful for cost optimization' is mild filler but earns its place by hinting at usage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter, annotation-free tool this covers each parameter acceptably, and the output schema exists so return-value detail is not required. Missing behavior-level notes (auth, default time window when start/end are omitted) keep it short of 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the prose must compensate, and it does: ISO 8601 format with an example, min_tokens threshold behavior, and the request-vs-response model distinction. The only shortfall is repeating the schema's default for limit without adding new meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Find traces with highest LLM token usage,' with a clear sort criterion (total token usage). However, it does not distinguish itself from near-identical siblings like get_llm_slow_traces or get_llm_usage, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Useful for cost optimization and identifying inefficient prompts' implies a use case but names no when-not conditions and no alternatives. With get_llm_slow_traces and get_llm_usage as obvious siblings, the lack of routing guidance is a real gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_llm_model_statsB

Get detailed performance statistics for a specific LLM model.

Analyzes request count, latency percentiles (p50, p95, p99), token usage statistics, error rates, and finish reason distributions.

Args: model_name: Model name to analyze (e.g., "gpt-4", "claude-3-opus", "gpt-3.5-turbo") start_time: Start time in ISO 8601 format (e.g., 2024-01-01T00:00:00Z) end_time: End time in ISO 8601 format service_name: Filter by service name

Returns: JSON string with comprehensive model statistics including duration/token percentiles

ParametersJSON Schema
NameRequiredDescriptionDefault
end_timeNo
model_nameYes
start_timeNo
service_nameNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the metrics computed and that a JSON string is returned, and the 'Get ... statistics' framing implies a read-only analysis. However it says nothing about cost of the query, permissions, or behavior when start_time/end_time are omitted, which matters for a query tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the purpose, then cleanly delimited Args/Returns blocks. The metric enumeration is slightly padded, but nothing is genuinely wasteful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so the Returns line need not document the response shape, and the description correctly focuses on purpose and input formats. It misses default time-window semantics and any note about how the result is scoped, which is a modest gap for a 4-parameter analytics tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does: all four parameters are named with formats (ISO 8601 for the time bounds) and concrete model-name examples. The gap is that it never states what a null/default time bound means (e.g. all time).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get detailed performance statistics for a specific LLM model') and enumerates the metrics covered (latency percentiles, token usage, error rates, finish reasons). It is clearly distinguishable from list_llm_models, but it does not explicitly contrast itself with the closest sibling get_llm_usage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance, no mention of prerequisites, and no routing to alternatives such as get_llm_usage or get_llm_slow_traces. Usage is only implied by the phrase 'for a specific LLM model.'

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_llm_slow_tracesA

Find slowest LLM traces by duration.

Useful for performance optimization and identifying latency bottlenecks.

Args: limit: Maximum number of traces to return (default: 10) start_time: Start time in ISO 8601 format (e.g., 2024-01-01T00:00:00Z) end_time: End time in ISO 8601 format min_duration_ms: Minimum duration threshold in milliseconds (only return traces above this) service_name: Filter by service name gen_ai_request_model: Filter by requested model name (e.g., "gpt-4") gen_ai_response_model: Filter by actual model used (e.g., "gpt-4-0613")

Returns: JSON string with top N slowest traces sorted by duration

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
end_timeNo
start_timeNo
service_nameNo
min_duration_msNo
gen_ai_request_modelNo
gen_ai_response_modelNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that results are sorted by duration and capped at a limit, which implies a read-only operation, but says nothing about permissions, rate limits, or whether traces are truncated. Adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded one-line purpose followed by use case and parameter list; the Args/Returns structure is scannable. The parameter block is somewhat long but each entry is terse and earns its place given 0% schema coverage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained in depth, and the description still notes the top-N-descending result shape. Combined with full parameter coverage and a mutation-free profile, an agent has enough to call this correctly; only sibling routing is left wanting.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate — and it does, documenting all 7 parameters with formats and examples (ISO 8601 for time, 'gpt-4' / 'gpt-4-0613' for the model params, default 10 for limit). It adds meaning the bare schema lacks, though it omits any clarification of the request vs. response model distinction.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource: 'Find slowest LLM traces by duration' — an agent immediately knows this returns traces ranked by latency. It distinguishes itself from get_llm_expensive_traces by emphasizing duration/latency rather than cost, though it never names that sibling explicitly to make the differentiation airtight.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Useful for performance optimization and identifying latency bottlenecks' implies the when-to-use context, but gives no explicit exclusions or alternatives. With siblings like get_llm_expensive_traces and find_errors in the same family, the agent must infer the routing itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_llm_usageB

Get aggregated LLM usage metrics (token counts) for a time period.

Provides breakdowns by model and service.

Args: start_time: Start time in ISO 8601 format end_time: End time in ISO 8601 format service_name: Filter by service name gen_ai_system: Filter by LLM provider gen_ai_request_model: Filter by requested model name gen_ai_response_model: Filter by actual model used limit: Maximum traces to analyze (default: 1000)

Returns: JSON string with usage metrics

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
end_timeNo
start_timeNo
service_nameNo
gen_ai_systemNo
gen_ai_request_modelNo
gen_ai_response_modelNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It discloses aggregation behavior, breakdown dimensions, and that limit controls traces analyzed, which helps. However, it does not state read-only safety, permissions, pagination, or other operational constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with purpose, then structured into Args and Returns. It is efficient overall, though the Returns line is somewhat redundant given that an output schema exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given seven optional filters and no annotations, the description covers the core operation and parameter meanings. It remains incomplete for agent routing because it lacks when-to-use guidance versus siblings and does not clarify optionality/defaults beyond limit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It documents all seven parameters with meanings, including ISO 8601 time formats, filter semantics, and the default limit of 1000. This adds substantial value beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Get aggregated LLM usage metrics (token counts) for a time period,' and adds breakdowns by model and service. It does not explicitly distinguish itself from similar siblings like get_llm_model_stats, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides no when-to-use guidance, no prerequisites, and no alternatives. It implies the tool is for usage metrics, but an agent must infer when to choose it over sibling tools such as get_llm_model_stats or get_llm_expensive_traces.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_traceB

Get complete trace details by trace ID.

Returns all spans with attributes, including parsed Opentelemetry data for LLM operations.

Args: trace_id: Trace identifier

Returns: JSON string with trace details

ParametersJSON Schema
NameRequiredDescriptionDefault
trace_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden, and it does disclose the payload shape (all spans with attributes, parsed OpenTelemetry data for LLM operations). It omits any read-only framing, error behavior for an unknown/expired trace ID, or size/truncation characteristics, which matters for a tool that returns an entire trace.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded in the first sentence and the rest is compact. The Args/Returns boilerplate duplicates what the schema and output schema already convey, costing a little efficiency.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so explaining return values is redundant but harmless, and the description's core claim (full trace with spans and parsed OTel data) is adequate. It leaves gaps around failure modes and the read-only/non-destructive nature that an agent would want before invoking a whole-trace retrieval.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description only restates the parameter as 'Trace identifier', adding no format, origin, or example information beyond the schema. With one parameter on which lookup success depends, this is under-specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb + resource ('Get complete trace details') and the retrieval key ('by trace ID'), which cleanly separates it from the search-oriented sibling search_traces. It never names a sibling explicitly, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus search_traces or search_spans_tool, and no prerequisite or exclusion conditions are stated. The 'by trace ID' phrasing implies a lookup use case, but the agent must infer that from the name and schema alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_llm_modelsB

List all LLM models being used with usage statistics.

Discovers what models are deployed and tracks their usage patterns.

Args: start_time: Start time in ISO 8601 format (e.g., 2024-01-01T00:00:00Z) end_time: End time in ISO 8601 format service_name: Filter by service name gen_ai_system: Filter by LLM provider (e.g., openai, anthropic, cohere) limit: Maximum traces to analyze for model discovery (default: 1000)

Returns: JSON string with list of models and their statistics (count, request_count, first_seen, last_seen)

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
end_timeNo
start_timeNo
service_nameNo
gen_ai_systemNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It usefully discloses the trace-scanning behavior via the limit parameter ('Maximum traces to analyze for model discovery') and describes the return shape (count, request_count, first_seen, last_seen), but says nothing about permissions, read-only status, or whether the trace scan is expensive/rate-limited.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the purpose before the Args/Returns blocks and uses clean sections. The second sentence ('Discovers what models are deployed and tracks their usage patterns') largely restates the first, a minor redundancy, but overall the text is well-sized and earns its space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values need not be re-explained, yet the description still summarizes them concisely. All five params are documented despite 0% schema coverage. The only real gap is behavioral context (no annotations, no permission or sibling-routing guidance), which keeps it short of a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does: each of the five parameters gets a meaning, ISO 8601 format guidance with an example, a provider example list (openai, anthropic, cohere), and the limit default. Format and filtering semantics are clear even though it does not explain null/default behavior for the optional filters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (list LLM models) plus scope ("being used with usage statistics") and adds the intent "discovers what models are deployed and tracks their usage patterns." However, it never distinguishes itself from the near-identical sibling get_llm_model_stats, so an agent cannot route between them from the description alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance, no prerequisites, and no mention of alternative tools. With siblings like get_llm_model_stats and get_llm_usage in the same family, the absence of any disambiguation leaves selection to guesswork; only the implicit 'discovery' framing hints at context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_llm_tools_toolA

List all LLM tools being used by identifying traceloop.span.kind == tool.

Discovers which tools/functions LLM applications are calling, grouped by tool name with usage statistics.

Args: start_time: Start time in ISO 8601 format (e.g., 2024-01-01T00:00:00Z) end_time: End time in ISO 8601 format service_name: Filter by service name gen_ai_system: Filter by LLM provider (openai, anthropic, etc.) limit: Maximum spans to analyze (default: 1000)

Returns: JSON string with list of tools and their statistics (usage count, services, first/last seen)

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
end_timeNo
start_timeNo
service_nameNo
gen_ai_systemNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden but only states the output format. It does not disclose whether the operation is read-only, whether it has side effects, permissions required, or performance characteristics beyond the limit parameter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured, front-loading the purpose, then listing args and returns. It is appropriately sized, though the Returns section is somewhat redundant given the output schema exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (5 optional params, output schema exists), the description covers purpose, parameters, and return format adequately. The main omission is usage guidance, but the core information needed to invoke the tool is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It provides meaningful details for each parameter, including ISO 8601 examples for start_time and end_time, filter semantics for service_name and gen_ai_system, and the default and purpose of limit.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('List all LLM tools being used') and explains the identifying mechanism ('traceloop.span.kind == tool'). It clearly distinguishes this from sibling tools like list_llm_models or search_traces by focusing on tool/function calls made by LLM applications.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool versus alternatives such as list_llm_models or get_llm_usage. The purpose implies usage, but no conditions or exclusions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_servicesA

List all available services in the OpenTelemetry backend.

Returns: JSON string with list of services

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden; it discloses the return type ('JSON string with list of services') but says nothing about ordering, pagination, authorization requirements, or failure behavior. For a no-argument read-only lister the risk is low, but the disclosure is thin.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short lines with the purpose front-loaded and no filler. The 'Returns' block is mildly redundant given an output schema exists, which keeps it off a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With zero parameters, an output schema present, and a trivial read operation, the definition covers what an agent needs to invoke it. The only real omission is guidance on when to reach for this tool versus the trace- and span-oriented siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so per the rubric this is the baseline 4. There are no arguments whose semantics the description would need to explain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List all available services') and scopes it to the OpenTelemetry backend, which is enough to separate it from sibling tools that operate on traces, spans, and LLM models. It does not explicitly name a sibling it differs from, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use statement and no named alternative, but the tool is a zero-argument discovery lister whose purpose implies the usage (enumerate services before filtering traces by service). Adequate but the definition never tells the agent to call this first or what to do with the result.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_spans_toolA

Search for individual OpenTelemetry spans with optional filters.

Unlike search_traces, this returns individual spans rather than grouped traces, which is useful for analyzing specific operations or finding spans with certain characteristics (e.g., LLM tool calls with traceloop.span.kind == tool).

Args: service_name: Filter by service name operation_name: Filter by operation/span name start_time: Start time in ISO 8601 format (e.g., 2024-01-01T00:00:00Z) end_time: End time in ISO 8601 format min_duration_ms: Minimum span duration in milliseconds max_duration_ms: Maximum span duration in milliseconds gen_ai_system: Filter by LLM provider (e.g., openai, anthropic) gen_ai_request_model: Filter by requested model name (e.g., "gpt-4") gen_ai_response_model: Filter by actual model used (e.g., "gpt-4-0613") has_error: Filter spans with errors tags: Additional tag filters as key-value pairs filters: Generic filter conditions - list of filter objects with: - field: Field name in dotted notation (e.g., "traceloop.span.kind") - operator: Comparison operator - value: Single value for most operators - values: List of values for "in", "not_in", "between" operators - value_type: Type of value(s) - "string", "number", or "boolean" limit: Maximum number of spans to return (1-1000, default: 100)

Returns: JSON string with span summaries

Example filter to find LLM tool calls: {"field": "traceloop.span.kind", "operator": "equals", "value": "tool", "value_type": "string"}

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNo
limitNo
filtersNo
end_timeNo
has_errorNo
start_timeNo
service_nameNo
gen_ai_systemNo
operation_nameNo
max_duration_msNo
min_duration_msNo
gen_ai_request_modelNo
gen_ai_response_modelNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It does disclose the limit range (1-1000, default 100) and the return format ('JSON string with span summaries'), but says nothing about pagination, ordering, or permission/auth requirements for a 13-parameter query tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded summary and the args list is well-organized and justified by 13 undocumented parameters, though the Returns and Example blocks add modest length beyond the essentials.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values needn't be spelled out, and the description covers all parameters plus the sibling comparison. It is nearly complete for a read-only search tool, with pagination/ordering behavior the main omission.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does: every one of the 13 parameters is documented with meaning, format (ISO 8601, dotted notation), examples ('gpt-4-0613', openai/anthropic), and the nested filter object's field/operator/value/value_type structure.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Search for individual OpenTelemetry spans') and explicitly differentiates from the sibling search_traces by explaining it returns individual spans rather than grouped traces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the alternative (search_traces) and the condition that selects this tool instead, plus a concrete use case (finding spans with traceloop.span.kind == tool). An agent can route correctly without opening either schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_tracesA

Search for OpenTelemetry traces with filters.

Supports both simple parameters and advanced generic filter system.

Args: service_name: Filter by service name (use filters for advanced queries) operation_name: Filter by operation/span name start_time: Start time in ISO 8601 format (e.g., 2024-01-01T00:00:00Z) end_time: End time in ISO 8601 format min_duration_ms: Minimum trace duration in milliseconds max_duration_ms: Maximum trace duration in milliseconds gen_ai_system: Filter by LLM provider (e.g., openai, anthropic) gen_ai_request_model: Filter by requested model name (e.g., gpt-4) gen_ai_response_model: Filter by actual model used (e.g., gpt-4-0613) has_error: Filter traces with errors tags: Additional tag filters as key-value pairs filters: Generic filter conditions (advanced) - list of filter objects with: - field: Field name in dotted notation (e.g., "gen_ai.usage.prompt_tokens") - operator: Comparison operator (equals, not_equals, gt, lt, gte, lte, contains, not_contains, starts_with, ends_with, in, not_in, between, exists, not_exists) - value: Single value for most operators - values: List of values for "in", "not_in", "between" operators - value_type: Type of value(s) - "string", "number", or "boolean" limit: Maximum number of traces to return (1-1000, default: 100)

Returns: JSON string with search results

Filter Examples: Find expensive traces: {"field": "gen_ai.usage.total_tokens", "operator": "gt", "value": 5000, "value_type": "number"}

Filter by multiple models:
{"field": "gen_ai.request.model", "operator": "in", "values": ["gpt-4", "claude-3"], "value_type": "string"}

Check if attribute exists:
{"field": "gen_ai.request.temperature", "operator": "exists", "value_type": "number"}

Find streaming requests:
{"field": "gen_ai.request.is_streaming", "operator": "equals", "value": true, "value_type": "boolean"}
ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNo
limitNo
filtersNo
end_timeNo
has_errorNo
start_timeNo
service_nameNo
gen_ai_systemNo
operation_nameNo
max_duration_msNo
min_duration_msNo
gen_ai_request_modelNo
gen_ai_response_modelNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It implies a read-only search and documents the limit default (100) and range (1-1000), but says nothing about ordering, pagination, or result shape beyond 'JSON string with search results'. Adequate but thin for a tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then a structured Args block and concrete examples. It is long, but given the 0% schema coverage the per-parameter documentation is earned rather than redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value detail is unnecessary, and the description covers all 13 params including the nested filter semantics. The remaining gap is the lack of sibling differentiation and pagination/ordering behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does: every one of the 13 parameters is documented in the Args block, and the complex 'filters' object is fully spelled out with field/operator/value/values/value_type and worked examples. Slightly marked down only because 'tags' is left vague.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (search OpenTelemetry traces) and adds a useful scope note about simple vs. generic filter modes. It does not, however, distinguish itself from near-siblings like search_spans_tool or find_errors, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives parameter-level guidance ('use filters for advanced queries') but no tool-level guidance about when to prefer this over find_errors, get_trace, or the LLM-specific trace helpers. Usage is implied by the filter list rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 11 tool updatesv0.1.0
    • First observedfind_errors
    • First observedget_llm_expensive_traces
    • First observedget_llm_model_stats
    • First observedget_llm_slow_traces
    • First observedget_llm_usage
    • First observedget_trace
    • First observedlist_llm_models
    • First observedlist_llm_tools_tool
    • First observedlist_services
    • First observedsearch_spans_tool
    • First observedsearch_traces

TDQS

A3.6/5.0

Scored across 11 tools

Disambiguation4/5

Most tools target distinct resources (traces, spans, services, models, tools, errors), and descriptions clarify boundaries. However, find_errors overlaps heavily with search_traces (has_error filter), and get_llm_expensive_traces/get_llm_slow_traces are specialized sorted variants of search_traces, which could cause misselection.

Naming Consistency4/5

The set largely follows a verb_noun pattern (get_trace, list_services, search_traces, find_errors, get_llm_model_stats, get_llm_expensive_traces). The main deviation is the awkward redundant '_tool' suffix on search_spans_tool and list_llm_tools_tool, breaking the otherwise clean convention.

Tool Count5/5

11 tools is well-scoped for an OpenTelemetry/LLM observability server, with each tool earning a distinct analytical purpose (search, aggregate, rank, discover). Neither thin nor bloated.

Completeness4/5

Strong read-only coverage across traces, spans, services, LLM usage/models/tools, and errors, which fits the observability domain. Minor gaps exist (no get_span by ID, no operations/dimension listing), but agents can work around them via search.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    C
    maintenance
    Connects AI assistants to Warpmetrics telemetry data to monitor AI agent performance, execution runs, and LLM costs. It allows users to query success rates, latency, and spend metrics directly through natural language interfaces.
    20
    25
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables natural language querying and analysis of OpenTelemetry traces, metrics, and logs stored in Elasticsearch/OpenSearch, allowing AI assistants to investigate performance issues, find root causes, and explore system behavior.
    21
    14
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    MCP server that gives AI agents access to your application's OpenTelemetry traces for querying, analysis, and debugging.
    5
    12
    2
    MIT