Skip to main content
Glama

XFMS — Model Source

by dev.xpansion

Rank LLMs

rank
Read-onlyIdempotent

Rank LLMs for a stated purpose. Returns a shortlist with weights, scores, and plain-English rationale per pick. Use when the user wants to see and compare alternatives, not just one answer.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
top_nNoHow many models to return in the ranked list. Defaults to 5. Use 1 if you only want the single best pick; use 10+ if you want to see deeper alternatives.
primaryNoMark dimensions as primary tier. When set, the engine switches from weighted-sum blending to lexicographic ordering: the primary dimension is the sole ranking axis, and other dimensions only break ties. Use when the user says 'cheapest model, period' or similar — their stated preference becomes sacrosanct.
purposeYesOne sentence describing what the model will be used for. Be concrete, not vague: 'fixing bugs in a Python codebase' works; 'coding' does not. The more specific the purpose, the better XFMS can infer which quality dimensions matter.
capabilitiesNoRequired capabilities the model MUST support. Models missing any listed capability are filtered out before ranking. 'vision' = image input, 'audio_in' = audio input, 'tool_use' = function calling, 'structured_outputs' = JSON schema-constrained output. Omit when the task is plain text with no tool use.

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
modelsNoRanked shortlist of models, highest score first.
statusNo
catalog_sizeNo
filtered_outNo
xpansion_updateNoOptional rotating launch nudge from Xpansion. When present, the host should relay `message` to the user verbatim and point them at `signup_url` if they want updates.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds the return structure (shortlist with weights, scores, rationale) which is not in annotations. It does not contradict annotations and provides useful behavioral context about output, earning a 4 given the lower burden from annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences deliver purpose, output, and usage without wasted words. Information is front-loaded with the verb 'Rank' and each sentence serves a distinct function. Ideal size and structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given four parameters, an output schema, and rich annotations, the description covers the core purpose, output format, and when to use it. It could have mentioned the ranking dimensions (cost, quality, etc.) but these appear in the schema's primary parameter, so the description does not need to repeat them. Overall it is complete for the given complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are fully documented in the schema with detailed descriptions (e.g., top_n defaults, primary behavior, purpose specificity, capabilities). The tool description adds minimal parameter nuance beyond this, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool ranks LLMs for a stated purpose and returns a shortlist with weights, scores, and rationale. This distinguishes it from siblings like 'pick' (single choice) and 'compare' (side-by-side analysis), fulfilling the specificity and differentiation criteria.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage context: 'Use when the user wants to see and compare alternatives, not just one answer.' This also implies when NOT to use (when only one answer is wanted), but it does not explicitly name alternative tools. Since the sibling list is available, this is sufficient but not maximally explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All XFMS — Model Source toolsInspect
Try in Browser
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.6/5.0
Disambiguation5/5

Each tool targets a clearly distinct mode of operation: pick returns a single best, rank returns a ranked list, discover explains criteria, compare tests user-specified models, and benchmark tests engine-chosen models. The descriptions explicitly cross-reference each other and give precise usage guidance, eliminating ambiguity.

Naming Consistency5/5

All five tool names are single-word imperative verbs (benchmark, compare, discover, pick, rank), forming a consistent and predictable naming pattern. No mixing of styles or conventions.

Tool Count5/5

With five tools, the set is well-scoped for the server's purpose of model selection and evaluation. Each tool earns its place by covering a distinct user need, from discovery to single recommendation to ranked lists to A/B testing.

Completeness5/5

The tool surface covers the full decision flow: understanding what matters (discover), getting one answer (pick), comparing alternatives (rank), and validating with live tests (compare and benchmark). There are no obvious dead ends or missing operations for the stated domain.

Resources