What can you do with this server?

The Atla MCP Server enables standardized evaluation of LLM responses using Atla's evaluation models, providing scores and textual critiques based on specific criteria. * Evaluate LLM responses: Score and critique responses against single or multiple evaluation criteria simultaneously * Choose evaluation models: Use flagship (atla-selene) or compact (atla-selene-mini) models * Customize evaluations: Optionally include original context and expected outputs * Integration options: Connect via OpenAI Agents SDK, Claude Desktop, or Cursor

Which integrations are available for this server?

Provides compatibility with the OpenAI Agents SDK, allowing users to connect to the Atla MCP server for LLM evaluation services.

1. Click on "Deploy Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Atla evaluate my response to 'Explain quantum computing' for accuracy and clarity" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

en es ja ko zh

Atla

Official

by atla-ai

Overview Schema Source Related Servers Discussions Admin

Python

Remote

Atla MCP Server

CAUTION

This repository was archived on July 21, 2025. The Atla API is no longer active.

An MCP server implementation providing a standardized interface for LLMs to interact with the Atla API for state-of-the-art LLMJ evaluation.

Learn more about Atla here. Learn more about the Model Context Protocol here.

Available Tools

evaluate_llm_response: Evaluate an LLM's response to a prompt using a given evaluation criteria. This function uses an Atla evaluation model under the hood to return a dictionary containing a score for the model's response and a textual critique containing feedback on the model's response.
evaluate_llm_response_on_multiple_criteria: Evaluate an LLM's response to a prompt across multiple evaluation criteria. This function uses an Atla evaluation model under the hood to return a list of dictionaries, each containing an evaluation score and critique for a given criteria.

Related MCP server: Ollama MCP Server

Usage

To use the MCP server, you will need an Atla API key. You can find your existing API key here or create a new one here.

Installation

We recommend using uv to manage the Python environment. See here for installation instructions.

Manually running the server

Once you have uv installed and have your Atla API key, you can manually run the MCP server using uvx (which is provided by uv):

ATLA_API_KEY=<your-api-key> uvx atla-mcp-server

Connecting to the server

Having issues or need help connecting to another client? Feel free to open an issue or contact us!

OpenAI Agents SDK

For more details on using the OpenAI Agents SDK with MCP servers, refer to the official documentation.

Install the OpenAI Agents SDK:

pip install openai-agents

Use the OpenAI Agents SDK to connect to the server:

import os

from agents import Agent
from agents.mcp import MCPServerStdio

async with MCPServerStdio(
        params={
            "command": "uvx",
            "args": ["atla-mcp-server"],
            "env": {"ATLA_API_KEY": os.environ.get("ATLA_API_KEY")}
        }
    ) as atla_mcp_server:
    ...

Claude Desktop

For more details on configuring MCP servers in Claude Desktop, refer to the official MCP quickstart guide.

Add the following to your claude_desktop_config.json file:

{
  "mcpServers": {
    "atla-mcp-server": {
      "command": "uvx",
      "args": ["atla-mcp-server"],
      "env": {
        "ATLA_API_KEY": "<your-atla-api-key>"
      }
    }
  }
}

Restart Claude Desktop to apply the changes.

You should now see options from atla-mcp-server in the list of available MCP tools.

Cursor

For more details on configuring MCP servers in Cursor, refer to the official documentation.

Add the following to your .cursor/mcp.json file:

{
  "mcpServers": {
    "atla-mcp-server": {
      "command": "uvx",
      "args": ["atla-mcp-server"],
      "env": {
        "ATLA_API_KEY": "<your-atla-api-key>"
      }
    }
  }
}

You should now see atla-mcp-server in the list of available MCP servers.

Contributing

Contributions are welcome! Please see the CONTRIBUTING.md file for details.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Available Tools

2 tools

evaluate_llm_responseA

Evaluate an LLM's response to a prompt using a given evaluation criteria.

This function uses an Atla evaluation model under the hood to return a dictionary
containing a score for the model's response and a textual critique containing
feedback on the model's response.

Returns:
    dict[str, str]: A dictionary containing the evaluation score and critique, in
        the format `{"score": <score>, "critique": <critique>}`.

Name	Required	Description	Default
`evaluation_criteria`	Yes	The specific criteria or instructions on which to evaluate the model output. A good evaluation criteria should provide the model with: (1) a description of the evaluation task, (2) a rubric of possible scores and their corresponding criteria, and (3) a final sentence clarifying expected score format. A good evaluation criteria should also be specific and focus on a single aspect of the model output. To evaluate a model's response on multiple criteria, use the `evaluate_llm_response_on_multiple_criteria` function and create individual criteria for each relevant evaluation task. Typical rubrics score responses either on a Likert scale from 1 to 5 or binary scale with scores of 'Yes' or 'No', depending on the specific evaluation task.
`llm_prompt`	Yes	The prompt given to an LLM to generate the `llm_response` to be evaluated.
`llm_response`	Yes	The output generated by the model in response to the `llm_prompt`, which needs to be evaluated.
`expected_llm_output`	No	A reference or ideal answer to compare against the `llm_response`. This is useful in cases where a specific output is expected from the model. Defaults to None.
`llm_context`	No	Additional context or information provided to the model during generation. This is useful in cases where the model was provided with additional information that is not part of the `llm_prompt` or `expected_llm_output` (e.g., a RAG retrieval context). Defaults to None.
`model_id`	No	The Atla model ID to use for evaluation. `atla-selene` is the flagship Atla model, optimized for the highest all-round performance. `atla-selene-mini` is a compact model that is generally faster and cheaper to run. Defaults to `atla-selene`.	atla-selene

Output Schema

Name	Required	Description
No output parameters

evaluate_llm_response_on_multiple_criteriaA

Evaluate an LLM's response to a prompt across multiple evaluation criteria.

This function uses an Atla evaluation model under the hood to return a list of
dictionaries, each containing an evaluation score and critique for a given
criteria.

Returns:
    list[dict[str, str]]: A list of dictionaries containing the evaluation score
        and critique, in the format `{"score": <score>, "critique": <critique>}`.
        The order of the dictionaries in the list will match the order of the
        criteria in the `evaluation_criteria_list` argument.

Name	Required	Description	Default
`evaluation_criteria_list`	Yes
`llm_prompt`	Yes	The prompt given to an LLM to generate the `llm_response` to be evaluated.
`llm_response`	Yes	The output generated by the model in response to the `llm_prompt`, which needs to be evaluated.
`expected_llm_output`	No	A reference or ideal answer to compare against the `llm_response`. This is useful in cases where a specific output is expected from the model. Defaults to None.
`llm_context`	No	Additional context or information provided to the model during generation. This is useful in cases where the model was provided with additional information that is not part of the `llm_prompt` or `expected_llm_output` (e.g., a RAG retrieval context). Defaults to None.
`model_id`	No	The Atla model ID to use for evaluation. `atla-selene` is the flagship Atla model, optimized for the highest all-round performance. `atla-selene-mini` is a compact model that is generally faster and cheaper to run. Defaults to `atla-selene`.	atla-selene

Output Schema

Name	Required	Description
`result`	Yes

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

2 tool updatesv1.0.0Jan 3, 2026
- Addedevaluate_llm_response
- Addedevaluate_llm_response_on_multiple_criteria

Deploy Server

Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

TDQS

A3.9/5.0

Scored 2026-05-04 00:00 across 2 tools

Disambiguation5/5

The two tools have clearly distinct purposes: one evaluates a single criterion, the other evaluates multiple criteria. Their names and descriptions make the difference unambiguous.

Naming Consistency5/5

Both tool names follow a consistent verb_noun_qualifier pattern, starting with 'evaluate_llm_response' and differentiating with '_on_multiple_criteria'. No mixing of conventions.

Tool Count3/5

With only 2 tools, the server feels thin. While the tools cover the core evaluation functionality, a typical well-scoped server has 3-15 tools, making this borderline insufficient.

Completeness3/5

The tools provide basic evaluation for single and multiple criteria, but lack supporting tools such as managing criteria, listing models, or retrieving history. The surface is minimal and may leave agents with limited options.

Maintenance

ActivityInactive

ResponsivenessOutdated data

Resources

Related MCP Connectors

mcpA
io.scorecard
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
RAG Systems Monitoring Testing & QA Tools
3
https://mcp.scorecard.io/mcp
anotherai
dev.anotherai
MCP server for building and testing AI agents with multi-model experimentation and insights.
Agent Orchestration Autonomous Agents Testing & QA Tools
2
https://api.anotherai.dev/mcp
mcp-aichat
io.github.AceDataCloud
MCP server for AI dialogue using various LLM models via AceDataCloud
AI & Machine Learning
2
https://aichat.mcp.acedata.cloud/mcp
mcpA
ai.llmse
Public MCP server for the LLM Search Engine
Search RAG Systems Developer Tools
102
https://llmse.ai/mcp

Related MCP Servers

mcp-llm
Code Execution Autonomous Agents
sammcj
A
license
B
quality
D
maintenance
An MCP server that provides LLMs access to other LLMs
Updated 2 years ago (2025-03-15 05:37 UTC)
4
38 npm
81
MIT
Ollama MCP Server
Agent Orchestration Autonomous Agents Code Execution
NewAITees
F
license
Not graded
quality
D
maintenance
A server that enables seamless integration between local Ollama LLM instances and MCP-compatible applications, providing advanced task decomposition, evaluation, and workflow management capabilities.
Updated 2 years ago (2025-03-02 23:19 UTC)
6
-
A2A Client MCP Server
Agent Orchestration Autonomous Agents
tesla0225
A
license
A
quality
D
maintenance
An MCP server that enables LLMs to interact with Agent-to-Agent (A2A) protocol compatible agents, allowing for sending messages, tracking tasks, and receiving streaming responses.
Updated 2 years ago (2025-04-13 07:21 UTC)
5
28
MIT
multivon-mcpofficial
AI & Machine Learning Testing & QA Tools
multivon-ai
A
license
A
quality
B
maintenance
MCP server that gives AI coding agents direct access to evaluation tools.
Updated 2 months ago (2026-09-17 01:56 UTC)
23
Apache 2.0

Atla MCP Server

Available Tools

Usage

Installation

Manually running the server

Connecting to the server

OpenAI Agents SDK

Claude Desktop

Cursor

Contributing

License

Available Tools

Output Schema

Output Schema

Tool Schema Changelog

TDQS

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

mcp-llm

Ollama MCP Server

A2A Client MCP Server

multivon-mcpofficial