LangSmith MCP Server

List LangSmith experiment projects (reference projects) with mandatory dataset filtering.

Fetches experiment projects from LangSmith that are associated with a specific dataset. These are projects used for model evaluation and comparison. Requires either a dataset ID or dataset name to filter experiments.

🧩 PURPOSE

This function provides a convenient way to list and explore LangSmith experiment projects. It supports:

• Filtering experiments by reference dataset (mandatory)

• Filtering projects by name (partial match)

• Limiting the number of results

• Automatically extracting deployment IDs from nested project data

• Returns simplified project information with key metrics (latency, cost, feedback stats)

⚙️ PARAMETERS

reference_dataset_id : str, optional The ID of the reference dataset to filter experiments by. Either this OR reference_dataset_name must be provided (but not both).

reference_dataset_name : str, optional The name of the reference dataset to filter experiments by. Either this OR reference_dataset_id must be provided (but not both).

limit : int, default 5 Maximum number of experiments to return. This can be adjusted by agents or users based on their needs.

project_name : str, optional Filter projects by name using partial matching. If provided, only projects whose names contain this string will be returned. Example: project_name="Chat" will match "Chat-LangChain", "ChatBot", etc.

📤 RETURNS

Dict[str, Any] A dictionary containing an "experiments" key with a list of simplified experiment project dictionaries:

```python
{
    "experiments": [
        {
            "name": "Experiment-Chat-LangChain",
            "experiment_id": "787d5165-f110-43ff-a3fb-66ea1a70c971",
            "feedback_stats": {...},  # Feedback statistics if available
            "latency_p50_seconds": 1.626,  # 50th percentile latency in seconds
            "latency_p99_seconds": 2.390,   # 99th percentile latency in seconds
            "total_cost": 0.00013005,       # Total cost in dollars
            "prompt_cost": 0.00002085,      # Prompt cost in dollars
            "completion_cost": 0.0001092,   # Completion cost in dollars
            "agent_deployment_id": "deployment-123"  # Only if available
        },
        ...
    ]
}
```

🧪 EXAMPLES

1️⃣ List experiments for a dataset by ID

experiments = list_experiments(reference_dataset_id="f5ca13c6-96ad-48ba-a432-ebb6bf94528f")

2️⃣ List experiments for a dataset by name

experiments = list_experiments(reference_dataset_name="my-dataset", limit=10)

3️⃣ Find experiments with specific name pattern

experiments = list_experiments(
    reference_dataset_id="f5ca13c6-96ad-48ba-a432-ebb6bf94528f",
    project_name="Chat",
    limit=1
)

🧠 NOTES FOR AGENTS

• Returns simplified experiment information with key metrics (latency, cost, feedback stats)

• The agent_deployment_id field is automatically extracted from nested project data when available, making it easy to identify agent deployments

• Experiments are filtered to include only reference projects (associated with datasets)

• The function uses name_contains for filtering, so partial matches work

• You must provide either reference_dataset_id OR reference_dataset_name, but not both

• Experiment projects are used for model evaluation and comparison across different runs