inspect-logs-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| INSPECT_LOG_DIR | No | Directory containing .eval log files | ./logs |
| INSPECT_LOGS_MCP_MAX_LIMIT | No | Maximum number of logs limit | 500 |
| INSPECT_LOGS_MCP_DEFAULT_LIMIT | No | Default number of logs limit | 50 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| tool_list_logsA | List available evaluation log files with metadata. Lists all .eval log files in the specified directory, sorted by date (newest first). Returns task name, model, status, sample count, and other metadata for each log. Args: log_dir: Directory containing log files. Defaults to INSPECT_LOG_DIR env var or ./logs limit: Maximum number of logs to return (default: INSPECT_LOGS_MCP_DEFAULT_LIMIT or 50, max: INSPECT_LOGS_MCP_MAX_LIMIT or 500) offset: Number of logs to skip for pagination (default: 0) |
| tool_get_eval_summaryC | Get comprehensive evaluation summary including header, results, and stats. Returns full metadata about an evaluation run: task info, model configuration, scoring results, token usage, duration, and revision info. Args: log_file: Path to log file (absolute or relative to log_dir) log_dir: Optional log directory for relative paths |
| tool_get_sampleB | Get detailed sample data including full conversation history. Returns the complete sample including input, target, all messages exchanged with the model, output, scores, and metadata. Args: log_file: Path to log file (absolute or relative to log_dir) sample_id: Sample ID to retrieve epoch: Epoch number (default: 1) log_dir: Optional log directory for relative paths include_events: Include event transcript (default: False, can be verbose) |
| tool_search_logsA | Search and filter evaluation logs by various criteria. Supports filtering by task name, model, status, date range, and minimum sample count. Task and model filters support wildcards (e.g., 'mind2web*', 'google/*'). Args: log_dir: Directory containing log files task: Filter by task name (supports wildcards like 'mind2web*') model: Filter by model name (supports wildcards like 'google/*') status: Filter by status: 'success', 'error', 'cancelled' date_from: Filter logs from this date (ISO format: YYYY-MM-DD) date_to: Filter logs until this date (ISO format: YYYY-MM-DD) min_samples: Minimum sample count limit: Maximum results (default: INSPECT_LOGS_MCP_DEFAULT_LIMIT or 50, max: INSPECT_LOGS_MCP_MAX_LIMIT or 500) |
| tool_compare_runsB | Compare metrics between two evaluation runs. Shows side-by-side comparison of two evaluation runs including task/model info, sample counts, score differences, token usage differences, and duration. Args: log_file_a: Path to first log file log_file_b: Path to second log file log_dir: Optional log directory for relative paths |
| tool_get_aggregate_statsA | Get aggregate statistics across multiple evaluation runs. Provides summary statistics grouped by task and model, including success rates, sample counts, token usage totals, and duration averages. Args: log_dir: Directory containing log files task: Filter by task name (supports wildcards) model: Filter by model name (supports wildcards) date_from: Filter logs from this date (ISO format) date_to: Filter logs until this date (ISO format) |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 6 tools
Each tool has a clearly distinct purpose with no overlap: compare_runs compares two specific runs, get_aggregate_stats provides cross-run statistics, get_eval_summary gives detailed metadata for a single run, get_sample retrieves granular sample data, list_logs enumerates available logs, and search_logs filters logs by criteria. The descriptions clearly differentiate their scopes, making misselection unlikely.
All tools follow a consistent verb_noun pattern with 'tool_' prefix (e.g., tool_compare_runs, tool_get_aggregate_stats). The verbs are descriptive and appropriate for their functions (compare, get, list, search), and the nouns precisely indicate the target resources (runs, stats, summary, sample, logs). This consistency aids predictability and usability.
With 6 tools, this server is well-scoped for inspecting evaluation logs. The count aligns with the domain's needs: listing and searching logs, retrieving summaries and samples, comparing runs, and aggregating statistics. Each tool earns its place without redundancy, covering essential operations without being overwhelming or insufficient.
The toolset provides complete coverage for log inspection: list_logs and search_logs for discovery, get_eval_summary and get_sample for detailed analysis, compare_runs for comparison, and get_aggregate_stats for cross-run insights. There are no obvious gaps; agents can navigate from discovery to deep analysis seamlessly, supporting typical evaluation workflows.