Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@aa-mcp List the top 5 LLMs by intelligence."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP server wrapping the Artificial Analysis public API. Enables AI agents to query LLM and multimodal model benchmarks, pricing, speed data, and track model updates via structured diffs.
The PyPI package is aa-mcp; it installs the aa-mcp console command.
Python 3.10+
uv (for installation and running)
An Artificial Analysis API key (get one free)
Related MCP server: LLM Benchmark MCP Server
Use uvx as the standard runtime path:
export ARTIFICIAL_ANALYSIS_API_KEY="aa_your_key_here"
uvx aa-mcpVariable | Required | Default | Description |
| Yes | - | Your AA API key |
| No |
| Directory for update snapshots |
| No |
| Log level (DEBUG, INFO, WARNING, ERROR) |
This server wraps the current free Artificial Analysis API endpoints documented at https://artificialanalysis.ai/api-reference:
Artificial Analysis endpoint | MCP tool |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
aa_list_llmsList LLM models with filtering and sorting.
Filters: creator, name, slug (substring match)
Sort by: intelligence (default), price, speed, ttft, coding, math
limit: Max results (default 20)
aa_get_modelGet full details for a single model by id, slug, or name.
Returns candidates if multiple matches found
Supports partial/fuzzy matching
aa_compare_modelsSide-by-side comparison of 2+ models.
Compares: intelligence, coding, math, pricing, speed, latency
Returns rankings across all metrics
Input: list of identifiers (ids, slugs, or names)
aa_list_recent_updatesDetect changes since the last local snapshot.
New models: present in current data but not in snapshot
Removed models: present in snapshot but gone from current data
Changed models: field-level diffs for pricing, speed, intelligence scores, etc.
First run creates a baseline snapshot
Float changes below 0.01 threshold are ignored (noise filtering)
aa_list_media_modelsQuery multimodal / media model rankings.
Modalities: text-to-image, image-editing, text-to-speech, text-to-video, image-to-video
top_n: Limit results (default 10)
include_categories: Per-category Elo breakdown where the upstream endpoint supports it
aa_evaluate_critptSubmit a complete CritPt benchmark batch to the official evaluation endpoint.
Requires submissions for the full public CritPt problem set
Validates required fields before sending: problem_id, generated_code, model, generation_config
Optional batch_metadata object is passed through to Artificial Analysis
The upstream endpoint is rate-limited separately and may take substantial time to complete
aa_healthcheckVerify API key and upstream connectivity.
Returns masked key preview, model count, rate limit info
Reports specific error types (auth, rate limit, server error)
The aa_list_recent_updates tool uses a local JSON snapshot mechanism:
First call: Fetches all LLM models, saves a normalized snapshot to disk, reports "baseline created"
Subsequent calls: Fetches fresh data, diffs against the latest snapshot, reports changes
Snapshot location: ~/.local/share/aa-mcp/snapshots/llm_models_YYYYMMDDTHHMMSSZ.json
Noise filtering: Float fields use a 0.01 threshold to avoid reporting insignificant fluctuations
Tracked fields: name, slug, creator, all evaluation scores, all pricing fields, speed/latency
Add to your opencode.json:
{
"mcp": {
"servers": {
"artificial-analysis": {
"command": "uvx",
"args": ["aa-mcp"],
"env": {
"ARTIFICIAL_ANALYSIS_API_KEY": "aa_your_key_here"
}
}
}
}
}For MCP client examples, see
docs/mcp-client-config.md.
# List top 5 most intelligent LLMs
aa_list_llms(sort_by="intelligence", limit=5)
# Get details on Claude 3.5 Sonnet
aa_get_model("claude-3-5-sonnet")
# Compare GPT-4o vs Claude 3.5 Sonnet vs Gemini 1.5 Pro
aa_compare_models(["gpt-4o", "claude-3-5-sonnet", "gemini-1.5-pro"])
# Check for recent model changes
aa_list_recent_updates()
# Top 5 text-to-image models
aa_list_media_models(modality="text-to-image", top_n=5)
# Submit CritPt benchmark results
aa_evaluate_critpt(
submissions=[
{
"problem_id": "Challenge_1_main",
"generated_code": "def solution(): return 42",
"model": "example-model",
"generation_config": {"temperature": 0}
}
],
batch_metadata={"run_id": "local-test"}
)
# Verify API connectivity
aa_healthcheck()For development, run the release checks from a source checkout:
uv sync --dev
uv run pytest
uv run ruff check .
uv build
uv run twine check dist/*Free API tier: 1000 requests/day rate limit
No explicit "updated_at" field: Update detection relies on snapshot diffs, not API metadata
LLM data only for snapshots: Media model snapshot tracking is not yet implemented
CritPt completeness: The upstream evaluation API requires submissions for the full public problem set; this server validates object shape but cannot verify set completeness locally
No pagination: The free API returns all models in a single response; no cursor/offset support
Snapshot storage: Local filesystem only; no cloud sync
This project uses data and benchmark resources from Artificial Analysis.
Attribution is required for all use of the Artificial Analysis free API. If you publish outputs, dashboards, reports, or derivative analysis using data returned by this MCP server, include attribution to artificialanalysis.ai.
CritPt benchmark evaluation data should also include attribution to the CritPt project.
Compare multiple LLM models side by side on key metrics.
Args:
identifiers: List of model ids, slugs, or names (at least 2).
Returns:
JSON object with comparison table and per-model details.
| Name | Required | Description | Default |
|---|---|---|---|
| identifiers | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description only states the return format ('comparison table and per-model details') without disclosing behavioral traits like read-only nature, side effects, or rate limits. For a comparison tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using two clear sentences plus structured Args/Returns. It is front-loaded with the purpose. Could be reduced to one line without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, return values are covered. The parameter is well-defined. Lacks context about error conditions or model availability, but for a simple comparison tool with one parameter it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description's Args section fully explains the parameter: 'List of model ids, slugs, or names (at least 2).' This adds essential meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Compare multiple LLM models side by side on key metrics,' which specifies the verb (compare), resource (LLM models), and scope (multiple). This distinguishes it from siblings like aa_get_model (single model) and aa_list_llms (list all).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., list vs compare). The description does not mention prerequisites or when not to use it, such as for large model lists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Submit a complete CritPt benchmark batch for official evaluation.
The upstream endpoint requires submissions for all public CritPt problems in
one request. Each submission should include problem_id, generated_code,
model, and generation_config. This tool can take substantial time because
the upstream grading service runs benchmark evaluation jobs.
Args:
submissions: Complete list of CritPt submission objects.
batch_metadata: Optional metadata for the batch.
Returns:
JSON response with accuracy, timeout rate, and judge/server error counts.
| Name | Required | Description | Default |
|---|---|---|---|
| submissions | Yes | ||
| batch_metadata | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description adds value by disclosing that the tool takes substantial time because the upstream grading service runs evaluation jobs. It also implies a write operation. No contradictions present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with a main paragraph followed by clear Args and Returns sections. It is informative without significant redundancy, though a bit verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool's purpose, usage constraints, parameter expectations, and return format are all covered. While the description assumes domain knowledge of CritPt and does not detail the submission object fields beyond names, it is largely complete for a submission tool with an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description compensates by specifying that each submission must include problem_id, generated_code, model, and generation_config, and that batch_metadata is optional. This adds meaningful structure beyond the schema's empty object types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool evaluates a complete CritPt benchmark batch, clearly distinguishing it from sibling tools like listing models or healthchecks. The verb 'submit' and resource 'CritPt benchmark batch' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that submissions must include all public CritPt problems in one request and warns about substantial runtime. However, it does not explicitly state when not to use this tool or name alternatives, though siblings are unrelated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get detailed information about a single LLM model by id, slug, or name.
Args:
identifier: Model id (UUID), slug, or name. Partial matching supported.
Returns:
JSON object with full model details, or candidates if multiple matches.
| Name | Required | Description | Default |
|---|---|---|---|
| identifier | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses partial matching and behavior on multiple matches (returns candidates). No annotations provided, so description carries full burden; it does so adequately but lacks mention of read-only status.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise with clear sections (Args, Returns). No unnecessary words, and layout helps parsing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has output schema, description covers return value (full model details or candidates). Single parameter, context is fully addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single parameter 'identifier' is explained beyond schema (UUID, slug, name; partial matching supported). Schema coverage is 0%, so description adds essential semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool retrieves detailed info about a single LLM model by identifier. It distinguishes from sibling tools like aa_list_llms (list all models) and aa_compare_models (compare models).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for fetching details of one model but does not explicitly contrast with siblings or state when not to use. No exclusion or alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verify API key validity and upstream API reachability.
Returns:
JSON with connectivity status, model count, and any error details.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description mentions it returns connectivity status, model count, and error details, but does not disclose whether it is safe to call repeatedly, if authentication is required (though implied by 'API key validity'), or any other behavioral traits beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: a single sentence for purpose followed by a bullet-like description of returns. No fluff, front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with no parameters and has an output schema. Description covers the key return fields (connectivity status, model count, error details), making it complete for a healthcheck tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in input schema, so baseline is 4. Description does not need to add param details. It adds meaning by explaining the return values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'Verify API key validity and upstream API reachability' with a clear verb and resource. It distinguishes from sibling tools like listing models or comparing models, which are different operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. Usage is implied as a preliminary check, but alternatives are not mentioned. Could specify that it is suitable for verifying connectivity before other operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List LLM models from Artificial Analysis with optional filtering and sorting.
Args:
creator: Filter by creator name (case-insensitive substring match)
name: Filter by model name (case-insensitive substring match)
slug: Filter by slug (case-insensitive substring match)
sort_by: Sort key - 'intelligence', 'price', 'speed', 'ttft', 'coding', 'math'
limit: Max models to return (default 20)
Returns:
JSON array of model summaries.
| Name | Required | Description | Default |
|---|---|---|---|
| creator | No | ||
| name | No | ||
| slug | No | ||
| sort_by | No | intelligence | |
| limit | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It discloses the filtering, sorting, and limit parameters, which are behavioral traits. However, it omits details like whether the operation is read-only (assumed but not stated), data freshness, rate limits, or error handling. Basic operational context is present but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise and well-structured. It opens with a clear purpose statement, lists parameters in a compact 'Args:' block with one-line explanations, and ends with a 'Returns:' note. Every sentence is relevant, and there is no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (5 optional parameters) and the presence of an output schema (not shown), the description covers the key inputs and return type. It lacks details on output fields, default sort order, and handling of no results, but these are often covered by the output schema. Minor gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate fully. It does so by explaining each parameter's purpose, filter behavior (case-insensitive substring match), and valid sort keys (intelligence, price, speed, etc.). This adds essential meaning beyond the schema's type and default values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists LLM models from Artificial Analysis with optional filtering and sorting. The verb 'list' and resource 'LLM models' are explicit, and it distinguishes itself from sibling tools like aa_get_model (single model) and aa_compare_models (comparison).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly guide when to use this tool vs alternatives. While the purpose implies it is for listing multiple models, there is no mention of when-not to use it or comparison with sibling tools like aa_get_model for single model retrieval. Usage context is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List top-ranked multimodal / media models by Elo ratings.
Args:
modality: One of 'text-to-image', 'image-editing', 'text-to-speech',
'text-to-video', 'image-to-video'
top_n: Number of top models to return (default 10)
include_categories: Include per-category Elo breakdown (text-to-image, text-to-video only)
Returns:
JSON array of media model rankings.
| Name | Required | Description | Default |
|---|---|---|---|
| modality | No | text-to-image | |
| top_n | No | ||
| include_categories | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It explains the tool returns a JSON array of rankings and describes parameters, but does not disclose read-only nature, authentication needs, rate limits, or data freshness. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, using a clean docstring format with Args and Returns sections. Every sentence provides value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 optional parameters, no required args, and an output schema existing, the description is nearly complete. It explains all parameters and return format. Minor gap: does not specify ordering direction (e.g., descending Elo) or any prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates. It explains each parameter: modality with five specific values, top_n with default 10, and include_categories with context that it applies only to text-to-image and text-to-video. Adds significant meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists top-ranked multimodal/media models by Elo ratings, specifying the verb 'List' and the resource. It naturally distinguishes from siblings like aa_list_llms (LLMs) and aa_get_model (specific model).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for using the tool—listing media model rankings—but does not explicitly state when not to use it or offer alternatives. No exclusions or comparative guidance given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Detect recent LLM model changes by comparing current data to the last local snapshot.
Identifies new models, removed models, and field-level changes (pricing, speed,
intelligence scores, etc.). On first run, saves a baseline snapshot and reports
the full model list as "initial baseline".
Args:
save_new_snapshot: If true (default), save the current data as the new snapshot
after diffing. Set false to preview-only.
Returns:
JSON with structured diff: added, removed, changed models with field-level deltas.
| Name | Required | Description | Default |
|---|---|---|---|
| save_new_snapshot | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the baseline snapshot creation on first run, the option to preview without saving, and the return format (structured diff). It does not cover potential side effects like disk usage or if the snapshot is stored persistently, but overall it provides sufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear overview paragraph followed by Args/Returns sections. It is informative without extraneous details. Minor improvement could be shortening the first sentence, but it's effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema is present but not shown in the context; however, the description covers return value structure ('JSON with structured diff'). The single parameter is fully explained. No gaps noted for a tool of this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. For the only parameter (save_new_snapshot), the description clearly explains its effect: saving after diffing vs preview-only. This goes well beyond the schema's default and type, fully compensating for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Detect') and resource ('LLM model changes by comparing current data to the last local snapshot'). It distinguishes from siblings like aa_compare_models (which compares specific models) and aa_list_llms (which lists all models) by focusing on change detection over time.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the initial snapshot behavior and preview mode via save_new_snapshot, giving implicit usage context. However, it does not explicitly state when to use this tool versus alternatives (e.g., aa_compare_models, aa_list_llms), nor does it mention prerequisites or clear 'when not to use' cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Each tool targets a distinct operation: healthcheck for connectivity, list_llms for listing models, get_model for details, compare_models for comparison, list_media_models for media models, list_recent_updates for tracking changes, and evaluate_critpt for benchmark submission. No overlap or ambiguity.
All tools use the 'aa_' prefix with consistent snake_case naming. Most follow a verb_noun pattern (list_llms, get_model, compare_models, evaluate_critpt, list_media_models, list_recent_updates), with only healthcheck being a single word, which is acceptable.
With 7 tools, the set is well-scoped for an LLM model comparison and evaluation API. It covers essential operations (listing, details, comparison, benchmark submission, updates tracking, healthcheck) without unnecessary bloat or missing core functionality.
The tool surface covers listing, detail, comparison, media models, updates, and evaluation submission. A minor gap is the lack of a tool to retrieve past evaluation results or browse problems, but the core workflow is supported.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Source-backed AI model pricing, rankings, history, and benchmark data.
Discover, compare, and monitor 1,400+ APIs directly from your AI coding agent.
Query your Betterlytics web analytics from AI agents: traffic, funnels, journeys, errors, uptime.
Give your agent web search and authoritative datasets: S&P Global, FRED, OECD, SimilarWeb & more.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Leev1s/aa-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
