Skip to main content
Glama
atla-ai

Atla

Official
by atla-ai
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
ATLA_API_KEYYesYour Atla API key, required to interact with the Atla API for LLM evaluation

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

This server publishes no instructions, or was last inspected before Glama recorded them.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tools
{
  "listChanged": false
}
prompts
{
  "listChanged": false
}
resources
{
  "subscribe": false,
  "listChanged": false
}
experimental
{}

Tools

Functions exposed to the LLM to take actions

NameDescription
evaluate_llm_responseA

Evaluate an LLM's response to a prompt using a given evaluation criteria.

This function uses an Atla evaluation model under the hood to return a dictionary
containing a score for the model's response and a textual critique containing
feedback on the model's response.

Returns:
    dict[str, str]: A dictionary containing the evaluation score and critique, in
        the format `{"score": <score>, "critique": <critique>}`.
evaluate_llm_response_on_multiple_criteriaA

Evaluate an LLM's response to a prompt across multiple evaluation criteria.

This function uses an Atla evaluation model under the hood to return a list of
dictionaries, each containing an evaluation score and critique for a given
criteria.

Returns:
    list[dict[str, str]]: A list of dictionaries containing the evaluation score
        and critique, in the format `{"score": <score>, "critique": <critique>}`.
        The order of the dictionaries in the list will match the order of the
        criteria in the `evaluation_criteria_list` argument.

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription

No resources

Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.9/5.0

Scored across 2 tools

Disambiguation5/5

The two tools have clearly distinct purposes: one evaluates a single criterion, the other evaluates multiple criteria. Their names and descriptions make the difference unambiguous.

Naming Consistency5/5

Both tool names follow a consistent verb_noun_qualifier pattern, starting with 'evaluate_llm_response' and differentiating with '_on_multiple_criteria'. No mixing of conventions.

Tool Count3/5

With only 2 tools, the server feels thin. While the tools cover the core evaluation functionality, a typical well-scoped server has 3-15 tools, making this borderline insufficient.

Completeness3/5

The tools provide basic evaluation for single and multiple criteria, but lack supporting tools such as managing criteria, listing models, or retrieving history. The surface is minimal and may leave agents with limited options.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources