Skip to main content
Glama

🐑 Shepherd MCP

MCP (Model Context Protocol) server for Shepherd - Debug your AI agents like you debug your code.

This MCP server allows AI assistants (Claude, Cursor, etc.) to query and analyze your AI agent sessions from multiple observability providers.

Supported Providers

  • AIOBS (Shepherd backend) - Native Shepherd observability

  • Langfuse - Open-source LLM observability platform

Related MCP server: chatlab-mcp

Installation

pip install shepherd-mcp

Or run directly with uvx:

uvx shepherd-mcp

Configuration

Environment Variables

AIOBS (Shepherd)

  • AIOBS_API_KEY (required) - Your Shepherd API key

  • AIOBS_ENDPOINT (optional) - Custom API endpoint URL

Langfuse

  • LANGFUSE_PUBLIC_KEY (required) - Your Langfuse public API key

  • LANGFUSE_SECRET_KEY (required) - Your Langfuse secret API key

  • LANGFUSE_HOST (optional) - Custom Langfuse host URL (defaults to cloud.langfuse.com)

.env File Support

shepherd-mcp automatically loads .env files from the current directory or any parent directory. This means if you have a .env file in your project root:

# .env
# AIOBS
AIOBS_API_KEY=aiobs_sk_xxxx

# Langfuse
LANGFUSE_PUBLIC_KEY=pk-lf-xxxx
LANGFUSE_SECRET_KEY=sk-lf-xxxx
LANGFUSE_HOST=https://cloud.langfuse.com

It will be automatically loaded when the MCP server starts.

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "shepherd": {
      "command": "uvx",
      "args": ["shepherd-mcp"],
      "env": {
        "AIOBS_API_KEY": "aiobs_sk_xxxx",
        "LANGFUSE_PUBLIC_KEY": "pk-lf-xxxx",
        "LANGFUSE_SECRET_KEY": "sk-lf-xxxx",
        "LANGFUSE_HOST": "https://cloud.langfuse.com"
      }
    }
  }
}

Cursor

Add to your .cursor/mcp.json:

{
  "mcpServers": {
    "shepherd": {
      "command": "uvx",
      "args": ["shepherd-mcp"],
      "env": {
        "AIOBS_API_KEY": "aiobs_sk_xxxx",
        "LANGFUSE_PUBLIC_KEY": "pk-lf-xxxx",
        "LANGFUSE_SECRET_KEY": "sk-lf-xxxx",
        "LANGFUSE_HOST": "https://cloud.langfuse.com"
      }
    }
  }
}

Or if installed via pip:

{
  "mcpServers": {
    "shepherd": {
      "command": "shepherd-mcp",
      "env": {
        "AIOBS_API_KEY": "aiobs_sk_xxxx",
        "LANGFUSE_PUBLIC_KEY": "pk-lf-xxxx",
        "LANGFUSE_SECRET_KEY": "sk-lf-xxxx",
        "LANGFUSE_HOST": "https://cloud.langfuse.com"
      }
    }
  }
}

Available Tools

AIOBS (Shepherd) Tools

aiobs_list_sessions

List all AI agent sessions from Shepherd.

Parameters:

  • limit (optional): Maximum number of sessions to return

Example prompt:

"List my recent AI agent sessions from AIOBS"

aiobs_get_session

Get detailed information about a specific session including the full trace tree, LLM calls, function events, and evaluations.

Parameters:

  • session_id (required): The UUID of the session to retrieve

Example prompt:

"Get AIOBS session details for abc123-def456"

aiobs_search_sessions

Search and filter sessions with multiple criteria.

Parameters:

  • query (optional): Text search (matches name, ID, labels, metadata)

  • labels (optional): Filter by labels as key-value pairs

  • provider (optional): Filter by LLM provider (e.g., 'openai', 'anthropic')

  • model (optional): Filter by model name (e.g., 'gpt-4o-mini', 'claude-3')

  • function (optional): Filter by function name

  • after (optional): Sessions started after date (YYYY-MM-DD)

  • before (optional): Sessions started before date (YYYY-MM-DD)

  • has_errors (optional): Only return sessions with errors

  • evals_failed (optional): Only return sessions with failed evaluations

  • limit (optional): Maximum number of sessions to return

Example prompts:

"Find all AIOBS sessions that used OpenAI with errors" "Search for sessions from yesterday that failed evaluations"

aiobs_diff_sessions

Compare two sessions and show their differences including:

  • Metadata: Duration, labels, timestamps

  • LLM calls: Count, tokens (input/output/total), average latency, errors

  • Provider/Model distribution: Which providers and models were used

  • Function events: Total calls, unique functions, function-specific counts

  • Trace structure: Trace depth, root nodes

  • Evaluations: Pass/fail counts and rates

  • System prompts: Compare system prompts across sessions

  • Request parameters: Temperature, max_tokens, tools used

  • Response content: Content length, tool calls, stop reasons

Parameters:

  • session_id_1 (required): First session UUID to compare

  • session_id_2 (required): Second session UUID to compare

Example prompt:

"Compare AIOBS sessions abc123 and def456"


Langfuse Tools

langfuse_list_traces

List traces with pagination and filters. Traces represent complete workflows or conversations.

Parameters:

  • limit (optional): Maximum results per page (default: 50)

  • page (optional): Page number (1-indexed)

  • user_id (optional): Filter by user ID

  • name (optional): Filter by trace name

  • session_id (optional): Filter by session ID

  • tags (optional): Filter by tags

  • from_timestamp (optional): Filter after timestamp

  • to_timestamp (optional): Filter before timestamp

Example prompt:

"List the last 20 Langfuse traces"

langfuse_get_trace

Get a specific trace with its observations (generations, spans, events).

Parameters:

  • trace_id (required): The trace ID to fetch

Example prompt:

"Get Langfuse trace details for trace-id-123"

langfuse_list_sessions

List sessions with pagination. Sessions group related traces together.

Parameters:

  • limit (optional): Maximum results per page

  • page (optional): Page number

  • from_timestamp (optional): Filter after timestamp

  • to_timestamp (optional): Filter before timestamp

Example prompt:

"Show me Langfuse sessions from the last week"

langfuse_get_session

Get a specific session with its metrics and traces.

Parameters:

  • session_id (required): The session ID to fetch

Example prompt:

"Get Langfuse session details for session-123"

langfuse_list_observations

List observations (generations, spans, events) with filters.

Parameters:

  • limit (optional): Maximum results per page

  • page (optional): Page number

  • name (optional): Filter by observation name

  • user_id (optional): Filter by user ID

  • trace_id (optional): Filter by trace ID

  • type (optional): Filter by type (GENERATION, SPAN, EVENT)

  • from_timestamp (optional): Filter after timestamp

  • to_timestamp (optional): Filter before timestamp

Example prompt:

"List all GENERATION type observations from Langfuse"

langfuse_get_observation

Get a specific observation with full details including input, output, usage, and costs.

Parameters:

  • observation_id (required): The observation ID to fetch

Example prompt:

"Get details for Langfuse observation obs-123"

langfuse_list_scores

List scores/evaluations with filters.

Parameters:

  • limit (optional): Maximum results per page

  • page (optional): Page number

  • name (optional): Filter by score name

  • user_id (optional): Filter by user ID

  • trace_id (optional): Filter by trace ID

  • from_timestamp (optional): Filter after timestamp

  • to_timestamp (optional): Filter before timestamp

Example prompt:

"Show me Langfuse scores for trace trace-123"

langfuse_get_score

Get a specific score/evaluation with full details.

Parameters:

  • score_id (required): The score ID to fetch

Example prompt:

"Get Langfuse score details for score-123"


Legacy Tools (Deprecated)

For backwards compatibility, the following tools are still available but will be removed in a future version:

  • list_sessions → Use aiobs_list_sessions

  • get_session → Use aiobs_get_session

  • search_sessions → Use aiobs_search_sessions

  • diff_sessions → Use aiobs_diff_sessions

Use Cases

1. Debugging Failed Runs

"Show me all AIOBS sessions that had errors in the last 24 hours"

2. Performance Analysis

"Compare AIOBS session abc123 with session def456 and tell me which one was more efficient"

3. Prompt Regression Detection

"Find Langfuse traces with failed evaluations"

4. Cost Tracking

"List Langfuse observations and summarize the total cost"

5. Session Inspection

"Get the full trace tree for the most recent Langfuse trace and explain what happened"

6. Cross-Provider Analysis

"Show me both AIOBS sessions and Langfuse traces from today"

Development

Setup

git clone https://github.com/neuralis/shepherd-mcp
cd shepherd-mcp
python -m venv venv
source venv/bin/activate
pip install -e ".[dev]"

Running Tests

pytest

Running Locally

export AIOBS_API_KEY=aiobs_sk_xxxx
export LANGFUSE_PUBLIC_KEY=pk-lf-xxxx
export LANGFUSE_SECRET_KEY=sk-lf-xxxx
python -m shepherd_mcp

Publishing to PyPI

Releases are automatically published to PyPI via GitHub Actions when a release is created.

To publish manually:

# Build the package
pip install build twine
python -m build

# Upload to PyPI
twine upload dist/*

Architecture

src/shepherd_mcp/
├── __init__.py          # Package exports
├── __main__.py          # Entry point
├── server.py            # MCP server with tool handlers
├── models/              # Data models
│   ├── __init__.py
│   ├── aiobs.py         # AIOBS-specific models
│   └── langfuse.py      # Langfuse-specific models
└── providers/           # Provider clients
    ├── __init__.py
    ├── base.py          # Base provider interface
    ├── aiobs.py         # AIOBS client implementation
    └── langfuse.py      # Langfuse client implementation
┌─────────────────┐     stdio      ┌─────────────────┐
│  Cursor/Claude  │ ◄────────────► │  shepherd-mcp   │
│    (Client)     │   stdin/stdout │   (subprocess)  │
└─────────────────┘                └────────┬────────┘
                                            │ HTTPS
                                  ┌─────────┴─────────┐
                                  │                   │
                                  ▼                   ▼
                         ┌─────────────┐     ┌─────────────┐
                         │ Shepherd API│     │ Langfuse API│
                         │   (AIOBS)   │     │   (Cloud)   │
                         └─────────────┘     └─────────────┘

License

MIT

Available Tools

18 tools
aiobs_diff_sessionsA

[AIOBS] Compare two AI agent sessions and show their differences including metadata, LLM calls, tokens, latency, providers, models, functions, evaluations, errors, system prompts, request parameters, and response content.

ParametersJSON Schema
NameRequiredDescriptionDefault
session_id_1YesFirst session UUID to compare
session_id_2YesSecond session UUID to compare

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. While it details what aspects are compared, it doesn't mention how differences are presented (e.g., structured output, visual diff), whether it's read-only or has side effects, performance considerations, or error handling for invalid session IDs. For a comparison tool with zero annotation coverage, this leaves significant behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the core purpose ('Compare two AI agent sessions and show their differences') followed by a comprehensive list of compared aspects. Every element earns its place by specifying the scope without redundancy, making it highly efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (comparing multiple session aspects) and lack of annotations/output schema, the description provides a good overview of what's compared but is incomplete. It doesn't cover behavioral traits (e.g., read-only nature, error handling) or output format, which are crucial for an AI agent to use it correctly. The description is adequate as a starting point but has clear gaps for full contextual understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with both parameters clearly documented as session UUIDs. The description doesn't add any parameter-specific details beyond what the schema provides (e.g., format examples, validation rules). According to scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in the description, which applies here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Compare two AI agent sessions and show their differences') and the comprehensive scope of comparison ('including metadata, LLM calls, tokens, latency, providers, models, functions, evaluations, errors, system prompts, request parameters, and response content'). It distinguishes this tool from sibling tools like aiobs_get_session (retrieves single session) and aiobs_list_sessions (lists multiple sessions) by focusing on comparative analysis.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through the phrase 'Compare two AI agent sessions,' suggesting this tool is for side-by-side analysis rather than individual session retrieval. However, it doesn't explicitly state when to use this versus alternatives like diff_sessions (a sibling with similar name) or provide exclusion criteria (e.g., when sessions are too dissimilar). The guidance is present but not explicit about alternatives or limitations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

aiobs_get_sessionB

[AIOBS] Get detailed information about a specific AI agent session including the full trace tree, LLM calls, function events, and evaluations.

ParametersJSON Schema
NameRequiredDescriptionDefault
session_idYesThe UUID of the session to retrieve

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While it describes what information is retrieved (trace tree, LLM calls, etc.), it doesn't mention important behavioral aspects like whether this is a read-only operation, what permissions might be required, whether there are rate limits, or what format the response takes. For a tool that retrieves detailed session data with no annotation coverage, this represents significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded with all necessary information in a single sentence. It efficiently communicates the tool's purpose and scope without any wasted words. The bracketed system identifier '[AIOBS]' is appropriately placed and doesn't interfere with readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (retrieving detailed session data), lack of annotations, and absence of an output schema, the description is minimally adequate but has clear gaps. It specifies what information is retrieved but doesn't describe the response format, error conditions, or behavioral constraints. For a tool with no output schema and no annotations, more contextual information would be helpful for an AI agent to use it effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the single parameter 'session_id' clearly documented as 'The UUID of the session to retrieve.' The description doesn't add any additional parameter information beyond what the schema provides, which is appropriate given the high schema coverage. The baseline score of 3 reflects adequate parameter documentation through the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get detailed information about a specific AI agent session' with specific components listed (trace tree, LLM calls, function events, evaluations). It distinguishes from siblings like 'aiobs_list_sessions' by focusing on a single session rather than listing multiple. However, it doesn't explicitly differentiate from 'get_session' or 'langfuse_get_session' which appear to be similar tools from different systems.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With multiple sibling tools like 'aiobs_list_sessions', 'aiobs_search_sessions', 'get_session', and 'langfuse_get_session', there's no indication of when this specific tool is appropriate versus those other options. The description only states what it does, not when to choose it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

aiobs_list_sessionsB

[AIOBS] List all AI agent sessions from Shepherd. Returns session metadata, labels, and event counts.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of sessions to return

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It mentions the return content but does not disclose behavioral traits such as pagination, rate limits, authentication needs, or whether it's read-only/destructive. For a list operation with no annotation coverage, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the purpose and output details. Every word earns its place, with no wasted information, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (one optional parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and output but lacks details on usage guidelines and behavioral transparency, which are needed for full completeness in this context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the single parameter 'limit' fully documented in the schema. The description does not add any parameter-specific details beyond what the schema provides, so it meets the baseline of 3 without compensating for gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List all AI agent sessions') and resource ('from Shepherd'), with specific output details ('session metadata, labels, and event counts'). It distinguishes from some siblings like 'aiobs_get_session' (singular) but not explicitly from 'list_sessions' or 'langfuse_list_sessions', which is why it's a 4 rather than a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. With many sibling tools (e.g., 'aiobs_search_sessions', 'langfuse_list_sessions', 'search_sessions'), the description lacks any context on use cases, prerequisites, or comparisons, leaving the agent to infer usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

aiobs_search_sessionsB

[AIOBS] Search and filter AI agent sessions with multiple criteria including text search, labels, provider, model, function name, date range, errors, and failed evaluations.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoText search query (matches session name, ID, labels, metadata)
labelsNoFilter by labels as key-value pairs (e.g., {"environment": "production"})
providerNoFilter by LLM provider (e.g., 'openai', 'anthropic')
modelNoFilter by model name (e.g., 'gpt-4o-mini', 'claude-3')
functionNoFilter by function name
afterNoSessions started after this date (YYYY-MM-DD or ISO format)
beforeNoSessions started before this date (YYYY-MM-DD or ISO format)
has_errorsNoOnly return sessions that have errors
evals_failedNoOnly return sessions with failed evaluations
limitNoMaximum number of sessions to return

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions the search/filter functionality, it doesn't describe important behaviors like pagination approach (implied by 'limit' parameter but not explained), return format, sorting behavior, error handling, or performance characteristics. For a search tool with 10 parameters, this leaves significant behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose. It could be slightly more structured by separating the purpose from the criteria list, but it avoids redundancy and wastes no words. Every element serves a purpose in conveying the tool's scope.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex search tool with 10 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what a 'session' represents in this context, what data is returned, how results are structured, or important behavioral aspects like pagination. The agent would need to guess about the return format and many operational details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly. The description adds minimal value beyond what's in the schema - it lists the criteria categories but doesn't provide additional semantic context like how 'query' search works (fuzzy vs exact), how labels filtering behaves, or date format details. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the verb ('Search and filter') and resource ('AI agent sessions'), and lists specific criteria (text search, labels, provider, etc.) that distinguish it from simpler list tools. It clearly communicates this is a multi-criteria search tool rather than a basic listing function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through the listed criteria (e.g., 'when you need to filter by provider, model, or errors'), but doesn't explicitly state when to use this tool versus alternatives like 'aiobs_list_sessions' or 'list_sessions'. No guidance is provided about when NOT to use it or specific prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

diff_sessionsB

[Deprecated: Use aiobs_diff_sessions] Compare two AI agent sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
session_id_1Yes
session_id_2Yes

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It mentions the tool is deprecated, which is useful behavioral context, but doesn't disclose what 'compare' entails (e.g., output format, whether it's read-only or has side effects, performance implications). For a tool with no annotations, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded: the deprecation warning comes first, followed by the core purpose. Every word earns its place with no redundancy, making it efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (comparing sessions), lack of annotations, no output schema, and 0% schema coverage, the description is incomplete. It doesn't explain what the comparison outputs, how sessions are identified, or any prerequisites. The deprecation note is helpful, but more context is needed for a deprecated tool that might still be invoked.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It doesn't mention parameters at all, failing to explain what session_id_1 and session_id_2 represent, their format, or how they should be provided. With two required parameters and no schema descriptions, this is inadequate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Compare two AI agent sessions.' It specifies the verb ('compare') and resource ('AI agent sessions'), and distinguishes it from siblings by indicating it's deprecated in favor of aiobs_diff_sessions. However, it doesn't fully differentiate from other comparison or session-related tools beyond the deprecation note.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: '[Deprecated: Use aiobs_diff_sessions]' directly tells the agent when not to use this tool and names the alternative. This is clear, actionable advice for tool selection, making it optimal for this dimension.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_sessionA

[Deprecated: Use aiobs_get_session] Get detailed information about a specific AI agent session.

ParametersJSON Schema
NameRequiredDescriptionDefault
session_idYesThe UUID of the session to retrieve

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It mentions the tool is deprecated, which is useful behavioral context, but does not disclose other traits like authentication needs, rate limits, or what 'detailed information' entails. The description adds some value but lacks comprehensive behavioral details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly concise and front-loaded, consisting of only two sentences that efficiently convey the deprecation warning and the tool's purpose. Every word serves a clear function, with no wasted information, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (single parameter, no output schema, no annotations), the description is somewhat complete by stating the deprecation and purpose. However, it lacks details on what 'detailed information' includes or behavioral aspects, leaving gaps that could hinder an agent's understanding of the tool's full context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the parameter 'session_id' fully documented as 'The UUID of the session to retrieve'. The description does not add any additional meaning beyond this, so it meets the baseline of 3 where the schema handles the parameter semantics effectively.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Get') and resource ('detailed information about a specific AI agent session'), and explicitly distinguishes it from its sibling tool 'aiobs_get_session' by marking it as deprecated. This provides precise differentiation and avoids confusion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidelines by stating '[Deprecated: Use aiobs_get_session]', which clearly indicates when not to use this tool and names the alternative. This direct guidance helps the agent avoid selecting an outdated tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_get_observationB

[Langfuse] Get a specific observation with full details including input, output, usage, and costs.

ParametersJSON Schema
NameRequiredDescriptionDefault
observation_idYesThe observation ID to fetch

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves data ('Get') and specifies the content included ('full details including input, output, usage, and costs'), which is helpful. However, it lacks critical behavioral details such as whether this is a read-only operation, error handling (e.g., for invalid IDs), authentication requirements, or rate limits. For a tool with no annotations, this leaves significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action ('Get a specific observation') and elaborates with essential details ('with full details including...'). There is no wasted text, and it directly communicates the tool's purpose without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (1 parameter, no nested objects) and high schema coverage (100%), the description is minimally adequate. However, with no annotations and no output schema, it fails to fully compensate for missing behavioral context (e.g., safety, errors) and output details. It provides basic purpose but lacks completeness for a tool that might involve data retrieval from an external service.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the single parameter 'observation_id' fully documented in the schema. The description adds no additional parameter semantics beyond implying that an observation ID is needed to fetch details. Since the schema already provides complete parameter information, the baseline score of 3 is appropriate, as the description doesn't enhance parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Get') and resource ('a specific observation') with scope ('with full details including input, output, usage, and costs'). It distinguishes from siblings like 'langfuse_list_observations' by specifying retrieval of a single observation rather than listing multiple. However, it doesn't explicitly contrast with other get tools like 'langfuse_get_session' or 'langfuse_get_trace'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites like needing an observation ID, nor does it differentiate from other get tools (e.g., 'langfuse_get_session' for session-level data) or list tools (e.g., 'langfuse_list_observations' for multiple observations). Usage is implied but not explicitly stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_get_scoreC

[Langfuse] Get a specific score/evaluation with full details.

ParametersJSON Schema
NameRequiredDescriptionDefault
score_idYesThe score ID to fetch

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool fetches 'full details' but doesn't specify what those details include, whether it's a read-only operation, error conditions (e.g., invalid score ID), or performance aspects like rate limits. This leaves significant gaps for a tool that retrieves data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the key action and resource. It avoids unnecessary words, though it could be slightly more structured by explicitly mentioning the parameter requirement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'full details' entails in the return value, error handling, or how this tool fits within the Langfuse ecosystem compared to siblings. For a data retrieval tool with no structured output documentation, this leaves the agent under-informed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with the 'score_id' parameter clearly documented in the schema. The description doesn't add any meaning beyond the schema (e.g., explaining what a score ID is or where to find it), so it meets the baseline of 3 for adequate but no extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('a specific score/evaluation with full details'), and the '[Langfuse]' prefix distinguishes it from non-Langfuse siblings. However, it doesn't explicitly differentiate from 'langfuse_list_scores' (which likely lists multiple scores) or 'langfuse_get_observation/session/trace' (which fetch different resource types), missing full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'langfuse_list_scores' for listing scores or other Langfuse get tools for different resources. It lacks context about prerequisites (e.g., needing a score ID) or exclusions, offering only a basic functional statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_get_sessionB

[Langfuse] Get a specific session with its metrics and aggregated data.

ParametersJSON Schema
NameRequiredDescriptionDefault
session_idYesThe session ID to fetch

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool fetches data ('Get'), implying a read-only operation, but doesn't clarify permissions, error handling, rate limits, or what happens if the session_id is invalid. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads key information: the tool name context ('[Langfuse]'), action, resource, and data scope. There is no wasted verbiage, making it highly concise and well-structured for quick comprehension.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (1 parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose but lacks details on usage context, behavioral traits, and output expectations. For a read operation with no annotations, it should provide more guidance on errors or data format to be fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the single parameter 'session_id' documented as 'The session ID to fetch'. The description adds no additional parameter semantics beyond this, such as format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate as the schema handles the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('a specific session'), and specifies the data included ('with its metrics and aggregated data'). It distinguishes from sibling tools like 'langfuse_list_sessions' by focusing on retrieval of a single session. However, it doesn't explicitly differentiate from other 'get_session' variants in the sibling list, which slightly limits specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid session_id), exclusions, or comparisons to sibling tools like 'langfuse_search_sessions' or 'langfuse_list_sessions', leaving the agent to infer usage context from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_get_traceB

[Langfuse] Get a specific trace with its observations. Returns full trace data including all observations (generations, spans, events).

ParametersJSON Schema
NameRequiredDescriptionDefault
trace_idYesThe trace ID to fetch

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states it returns full trace data including observations, but doesn't mention authentication requirements, rate limits, error conditions, or whether this is a read-only operation. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose and includes important details about what's returned. Every word serves a purpose with zero waste or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read operation with one parameter and no output schema, the description provides adequate context about what the tool does and returns. However, without annotations covering behavioral aspects like authentication or safety, and no output schema to describe the return format, there are completeness gaps that could hinder effective tool selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100% with one parameter clearly documented, so the baseline is 3. The description doesn't add any parameter-specific information beyond what's in the schema, but doesn't need to compensate for coverage gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get a specific trace') and resource ('trace with its observations'), specifying it returns full trace data including all observations. It distinguishes from siblings like 'langfuse_list_traces' by focusing on a single trace, but doesn't explicitly contrast with 'langfuse_get_session' or 'langfuse_get_observation'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you need a specific trace with its observations, but doesn't provide explicit guidance on when to use this versus alternatives like 'langfuse_list_traces' for multiple traces or 'langfuse_get_observation' for individual observations. No exclusions or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_list_observationsC

[Langfuse] List observations (generations, spans, events) with filters. Observations are the building blocks of traces.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of results per page (default: 50)
pageNoPage number (1-indexed, default: 1)
nameNoFilter by observation name
user_idNoFilter by user ID
trace_idNoFilter by trace ID
typeNoFilter by observation type
from_timestampNoFilter observations starting after this timestamp
to_timestampNoFilter observations starting before this timestamp

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions filtering and pagination indirectly via parameters, but lacks details on rate limits, authentication needs, error handling, or response format. This is insufficient for a list operation with 8 parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is brief and front-loaded, with two sentences that efficiently state the purpose and context. However, it could be more structured by explicitly mentioning pagination or filtering scope.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of 8 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain the return format, pagination behavior, or error cases, leaving gaps for the agent to infer usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 8 parameters. The description adds minimal value by mentioning 'filters' but doesn't explain parameter interactions or provide examples. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'List observations (generations, spans, events) with filters.' It specifies the verb ('List') and resource ('observations'), and distinguishes them as 'building blocks of traces.' However, it doesn't explicitly differentiate from sibling tools like 'langfuse_list_sessions' or 'langfuse_list_traces,' which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It mentions filters but doesn't specify scenarios or prerequisites, nor does it reference sibling tools for comparison. This leaves the agent without context for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_list_scoresC

[Langfuse] List scores/evaluations with filters. Scores are attached to traces or observations.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of results per page (default: 50)
pageNoPage number (1-indexed, default: 1)
nameNoFilter by score name
user_idNoFilter by user ID
trace_idNoFilter by trace ID
from_timestampNoFilter scores created after this timestamp
to_timestampNoFilter scores created before this timestamp

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that scores are attached to traces or observations, adding some context, but fails to describe key behaviors such as pagination handling (implied by limit/page parameters), rate limits, authentication needs, or what the output looks like. This is inadequate for a list tool with 7 parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action ('List scores/evaluations') and includes essential context ('with filters', 'attached to traces or observations'). There is zero waste, making it appropriately sized and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a list tool with 7 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on behavioral traits, output format, and usage guidelines, making it insufficient for an agent to fully understand how to invoke and interpret results from this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 7 parameters. The description adds no additional meaning beyond what the schema provides, such as explaining relationships between filters or usage examples. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('List') and resource ('scores/evaluations') with the context of filtering, making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'langfuse_get_score' or 'langfuse_search_sessions', which might handle similar data but with different operations or scopes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions 'with filters' but provides no guidance on when to use this tool versus alternatives like 'langfuse_search_sessions' or 'langfuse_list_traces'. It lacks explicit when/when-not instructions or prerequisites, leaving usage context implied at best.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_list_sessionsC

[Langfuse] List sessions with pagination. Sessions group related traces together.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of results per page (default: 50)
pageNoPage number (1-indexed, default: 1)
from_timestampNoFilter sessions created after this timestamp
to_timestampNoFilter sessions created before this timestamp

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It mentions pagination behavior, which is useful. However, it doesn't disclose other important behavioral traits: whether this is a read-only operation, what authentication is required, rate limits, error conditions, or what the response format looks like (especially critical since there's no output schema). For a tool with 4 parameters and no annotations, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise - just two sentences that each earn their place. The first sentence states the core functionality with pagination, and the second provides valuable context about what sessions represent. There's zero waste or redundancy, and it's appropriately front-loaded with the main purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (4 parameters, no annotations, no output schema, multiple similar sibling tools), the description is incomplete. It doesn't explain the return format, error handling, authentication requirements, or when to use versus search alternatives. For a list operation with pagination and timestamp filtering, more context about response structure and usage scenarios would be needed for an agent to use it effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all 4 parameters with their types and descriptions. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain timestamp format, pagination behavior details, or relationships between parameters. With complete schema coverage, baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('List') and resource ('sessions'), and specifies that it includes pagination. It distinguishes from siblings by mentioning that sessions group related traces together, which provides context about the resource type. However, it doesn't explicitly differentiate from other list/search tools in the sibling set (like langfuse_search_sessions or list_sessions).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With multiple similar tools in the sibling set (langfuse_search_sessions, search_sessions, list_sessions), there's no indication of when this paginated list is preferred over search functionality or other listing tools. No prerequisites, exclusions, or alternative recommendations are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_list_tracesA

[Langfuse] List traces with pagination and filters. Traces represent complete workflows or conversations.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of results per page (default: 50)
pageNoPage number (1-indexed, default: 1)
user_idNoFilter by user ID
nameNoFilter by trace name
session_idNoFilter by session ID
tagsNoFilter by tags
from_timestampNoFilter traces starting after this timestamp (ISO format or YYYY-MM-DD)
to_timestampNoFilter traces starting before this timestamp (ISO format or YYYY-MM-DD)

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden. It discloses pagination behavior and filtering capabilities, which is helpful. However, it doesn't mention authentication requirements, rate limits, error conditions, or what the response structure looks like (especially important since there's no output schema). For a list operation with 8 parameters, more behavioral context would be beneficial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise - just two sentences that efficiently convey the core functionality. The first sentence states the action with key behavioral aspects (pagination, filters). The second sentence provides helpful context about what traces represent. Every word earns its place with zero redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters, no annotations, and no output schema, the description provides basic but incomplete context. It covers the what (list traces) and some how (pagination, filters), but lacks information about authentication, error handling, response format, and performance characteristics. For a tool with this complexity and no structured output documentation, the description should do more to help the agent understand what to expect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly with descriptions and format hints. The description adds no additional parameter information beyond mentioning 'pagination and filters' generally. This meets the baseline of 3 when schema does the heavy lifting, but doesn't provide extra value like explaining parameter interactions or constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List traces') and resource ('traces'), and explains what traces represent ('complete workflows or conversations'). It distinguishes from siblings like 'langfuse_get_trace' (singular) and 'langfuse_search_traces' (search vs list). However, it doesn't explicitly differentiate from 'langfuse_list_sessions' or 'langfuse_list_observations' which are different resource types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through 'with pagination and filters' and the explanation of traces, suggesting this is for retrieving multiple traces with optional filtering. However, it doesn't explicitly state when to use this versus alternatives like 'langfuse_search_traces' (which might offer different search capabilities) or 'langfuse_get_trace' (for single trace). No explicit when-not-to-use guidance is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_search_sessionsA

[Langfuse] Search and filter sessions with extended criteria including text search, user ID, trace count range, and cost range. Combines API-level and client-side filtering.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoText search query (matches session ID or user IDs)
user_idNoFilter by user ID (client-side)
min_tracesNoMinimum number of traces in session (client-side filter)
max_tracesNoMaximum number of traces in session (client-side filter)
min_costNoMinimum total cost (client-side filter)
max_costNoMaximum total cost (client-side filter)
from_timestampNoFilter sessions created after this timestamp
to_timestampNoFilter sessions created before this timestamp
limitNoMaximum number of results (default: 50)
pageNoPage number (1-indexed, default: 1)

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'combines API-level and client-side filtering' which provides some implementation insight, but doesn't cover important aspects like pagination behavior (implied by limit/page params), rate limits, authentication requirements, error conditions, or what 'client-side' filtering entails operationally.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly concise - two sentences that efficiently convey the tool's purpose and key differentiators. Every word earns its place, with no redundant information or unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter search tool with no annotations and no output schema, the description provides adequate purpose and filtering approach context. However, it lacks information about return format, result ordering, error handling, and the practical implications of 'client-side filtering' which would be important for proper tool selection and usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly. The description adds marginal value by grouping parameters conceptually ('text search, user ID, trace count range, and cost range') and distinguishing API vs. client-side filters, but doesn't provide additional semantic context beyond what's in the parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('search and filter sessions') and resources ('sessions'), and distinguishes it from siblings by mentioning 'extended criteria' and 'combines API-level and client-side filtering' which differentiates it from simpler list/search tools in the sibling set.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through 'extended criteria' and the combination of filtering approaches, suggesting this is for more complex session searches. However, it doesn't explicitly state when to use this vs. simpler alternatives like 'langfuse_list_sessions' or 'search_sessions' from the sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

langfuse_search_tracesC

[Langfuse] Search and filter traces with extended criteria including text search, release, cost range, and latency range. Combines API-level and client-side filtering.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNoText search query (matches trace name, ID, user ID, session ID, or tags)
nameNoFilter by trace name (API-level)
user_idNoFilter by user ID
session_idNoFilter by session ID
tagsNoFilter by tags
releaseNoFilter by release (client-side)
min_costNoMinimum total cost (client-side filter)
max_costNoMaximum total cost (client-side filter)
min_latencyNoMinimum latency in seconds (client-side filter)
max_latencyNoMaximum latency in seconds (client-side filter)
from_timestampNoFilter traces starting after this timestamp (ISO format or YYYY-MM-DD)
to_timestampNoFilter traces starting before this timestamp (ISO format or YYYY-MM-DD)
limitNoMaximum number of results (default: 50)
pageNoPage number (1-indexed, default: 1)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions filtering capabilities but lacks critical details: it doesn't specify if this is a read-only operation (likely, but not stated), whether it has pagination behavior (implied by 'limit' and 'page' parameters but not described), rate limits, authentication needs, or what the output format looks like (no output schema). The description adds some context about API vs. client-side filtering but misses essential behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately concise with two sentences. The first sentence clearly states the purpose and key criteria, and the second adds context about filtering types. There's no wasted text, and it's front-loaded with essential information. A perfect score would require more comprehensive guidance without sacrificing brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (14 parameters, no annotations, no output schema), the description is incomplete. It adequately explains what the tool does but fails to provide sufficient context on usage guidelines, behavioral aspects like pagination or safety, and output expectations. For a search tool with many parameters and no structured output information, more detail is needed to help an agent use it effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds minimal parameter semantics beyond the schema. It lists examples of criteria (text search, release, cost range, latency range) which are covered in the schema descriptions. With 100% schema description coverage, the baseline is 3, as the schema already documents all 14 parameters thoroughly. The description doesn't provide additional syntax, format details, or usage examples for parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Search and filter traces with extended criteria including text search, release, cost range, and latency range.' It specifies the verb (search/filter) and resource (traces) with examples of criteria. However, it doesn't explicitly differentiate from sibling tools like 'langfuse_list_traces' or 'langfuse_search_sessions,' which would be needed for a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides minimal usage guidance. It mentions that it 'combines API-level and client-side filtering,' which hints at its capabilities but doesn't specify when to use this tool versus alternatives like 'langfuse_list_traces' (which might be for simpler listing) or 'langfuse_search_sessions' (for different resources). No explicit when-to-use or when-not-to-use scenarios are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_sessionsA

[Deprecated: Use aiobs_list_sessions] List all AI agent sessions from Shepherd.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of sessions to return

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a list operation but doesn't describe what 'list all' means in practice (pagination behavior, default ordering, what fields are returned, or any limitations). For a list tool with no annotations, this leaves significant behavioral questions unanswered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Extremely concise - just one sentence that serves two purposes: deprecation warning and functional description. Every word earns its place, and the critical deprecation information is front-loaded with brackets.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one documented parameter and no output schema, the description provides minimal but adequate context about what the tool does and its deprecation status. However, it lacks information about return format, pagination, or any behavioral constraints that would be helpful for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the single 'limit' parameter. The description doesn't add any parameter-specific information beyond what the schema provides. Baseline 3 is appropriate when the schema does the documentation work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('List') and resource ('all AI agent sessions from Shepherd'), making the purpose understandable. However, it doesn't distinguish this tool from its many siblings beyond the deprecation note, which slightly reduces clarity for current usage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when NOT to use this tool ('[Deprecated: Use aiobs_list_sessions]'), naming a specific alternative. This is perfect guidance for tool selection, even though it's negative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_sessionsC

[Deprecated: Use aiobs_search_sessions] Search and filter AI agent sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryNo
labelsNo
providerNo
modelNo
functionNo
afterNo
beforeNo
has_errorsNo
evals_failedNo
limitNo

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'search and filter,' implying a read-only operation, but doesn't describe traits like pagination, rate limits, authentication needs, or what happens with no results. The deprecation note adds some context, but overall, behavioral details are insufficient for a tool with 10 parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded with the deprecation warning, followed by the core purpose. It uses two short sentences with no wasted words, making it easy to parse. However, it could be more structured by separating deprecation from usage instructions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (10 parameters, no schema descriptions, no annotations, no output schema), the description is incomplete. It doesn't explain return values, error handling, or how parameters affect results. The deprecation note adds some context, but overall, it's inadequate for guiding an agent in selecting and invoking this tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate by explaining parameters. It only mentions 'search and filter' generically, without detailing what 'query,' 'labels,' 'provider,' etc., mean or how they interact. With 10 undocumented parameters, this adds minimal value beyond the schema, failing to clarify semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool's purpose as 'Search and filter AI agent sessions,' which is clear but vague. It doesn't specify what resources or data are searched (e.g., session metadata, logs) or how filtering works, and it doesn't distinguish from siblings like 'list_sessions' or 'aiobs_search_sessions' beyond the deprecation note. This is a basic statement of function without specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides minimal guidance: it explicitly states '[Deprecated: Use aiobs_search_sessions]' to indicate an alternative, but offers no context on when to use this tool versus other siblings like 'list_sessions' or 'langfuse_search_sessions.' It lacks information on prerequisites, use cases, or exclusions, leaving the agent with little direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

B3.1/5.0
Disambiguation3/5

The tools are organized into two clear domains (AIOBS and Langfuse), but within each domain there is significant overlap. For example, aiobs_list_sessions and aiobs_search_sessions both retrieve sessions, with the latter adding filtering, which could cause confusion about which to use. Similarly, langfuse_list_sessions and langfuse_search_sessions serve very similar purposes. The deprecated tools further add redundancy without clear differentiation from their AIOBS counterparts.

Naming Consistency4/5

The naming follows a consistent prefix_verb_noun pattern (e.g., aiobs_diff_sessions, langfuse_get_observation), which is predictable and readable. However, there are minor deviations: some tools use 'list' while others use 'search' for similar operations, and the deprecated tools lack the prefix, breaking full consistency but not severely impacting usability.

Tool Count3/5

With 18 tools, the count is borderline high for the scope, which appears to be monitoring and analyzing AI agent sessions and traces. The inclusion of deprecated tools (6 out of 18) inflates the number unnecessarily, making the set feel heavy and cluttered, though the core functionality is well-represented.

Completeness4/5

The tool set provides comprehensive coverage for querying and analyzing sessions, traces, observations, and scores across two systems (AIOBS and Langfuse), with operations like get, list, search, and diff. Minor gaps might include update or delete operations, but these are likely not needed for the monitoring domain, and agents can work effectively with the provided read-oriented tools.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    C
    maintenance
    Connects AI assistants to Warpmetrics telemetry data to monitor AI agent performance, execution runs, and LLM costs. It allows users to query success rates, latency, and spend metrics directly through natural language interfaces.
    20
    59
    MIT
  • F
    license
    A
    quality
    F
    maintenance
    Enables AI assistants to query local ChatLab chat history using natural language, with tools for session listing, message retrieval, and analytics.
    17
    16
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI assistants to interact with Datadog's observability platform via natural language, covering metrics, logs, APM, monitors, dashboards, incidents, and infrastructure.
    1,106
    1
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables natural language querying and analysis of OpenTelemetry traces, metrics, and logs stored in Elasticsearch/OpenSearch, allowing AI assistants to investigate performance issues, find root causes, and explore system behavior.
    16
    14
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/neuralis-in/shepherd-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server