Skip to main content
Glama

Pareto trade-off frontier

aa_pareto

Find Pareto-optimal AI models by trading off any score against cost, size, speed, or latency, then identify the knee point and frontier coverage.

Instructions

The Pareto frontier: any score vs any cost/size/speed axis, with the knee.

Always available — no per-benchmark tool needed. Returns the models where no other model has both a lower-or-equal x and a higher-or-equal score, ordered by x ascending, plus a knee line and coverage.

The three site views this covers:

  • aa_pareto (default) = "Intelligence Index vs. Cost per Index Task"

  • aa_pareto x=out_tokens_per_task = "Intelligence Index vs. Output Tokens per Task"

  • aa_pareto x=index_total_cost = "Intelligence Index vs. Cost to Run the AA Intelligence Index" (whole suite)

Args: score: any rankable with per-model numbers (default intelligence); e.g. cyberIndex, terminalBench40, gpqa, capabilities.legal, openness. x: the trade-off axis — cost_per_task (default), index_total_cost, out_tokens_per_task, time_per_task, context_tokens, params_b, ttft, price_in, price_out. top: frontier points to return (default 15; cheapest first). creator: substring match on the creator/lab. open_weights: True/False to keep only open- or closed-weights models. min_intelligence: drop models below this Intelligence Index value. min_context: drop models with a smaller context window (tokens). include_deprecated: include deprecated rows (default False). include_all: return every frontier point instead of the top cheapest. refresh: bypass the cache and re-fetch.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
xNocost_per_task
topNo
scoreNointelligence
creatorNo
refreshNo
include_allNo
min_contextNo
open_weightsNo
min_intelligenceNo
include_deprecatedNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations the description carries the full burden and does well: it defines the dominance rule, the ordering (x ascending), the default cheapest-first truncation, that deprecated rows are excluded by default, and that refresh bypasses the cache. It does not state freshness/rate-limit behavior or the exact response shape, but the cached-vs-refreshed distinction and default filtering are real disclosures.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core definition and dominance rule, then organized into site-view mapping and Args sections so an agent can scan it. The site-view block is slightly verbose for three examples and the smart-quote formatting is noisy, but no sentence is purely filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description does explain what is returned (non-dominated models, x-ascending order, knee line, coverage) as well as default truncation and filtering. It stops short of describing the concrete response structure or error behavior, leaving a small gap for a 10-parameter analytical tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and all 10 parameters are undocumented in the schema, yet the Args block defines every one, including enumerated values for x (cost_per_task, index_total_cost, out_tokens_per_task, time_per_task, context_tokens, params_b, ttft, price_in, price_out) and example values for score. This is exactly the compensation the zero-coverage schema requires.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb/resource (compute the Pareto frontier of any score against any trade-off axis) and quantifies the output (frontier points plus a knee and coverage). This is clearly distinguishable from aa_leaderboard or aa_compare, which rank rather than compute a non-dominated set.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells the agent it is 'Always available — no per-benchmark tool needed' and maps three concrete site views onto parameter combinations, which is strong when-to-use context. It never excludes or routes against specific siblings such as aa_compare, so it stops short of full 5-level routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.