Skip to main content
Glama

Get the model benchmark

get_benchmark
Read-onlyIdempotent

Which model to use for a job, in three tiers (best, middle, cheapest), priced for this account: credits per reference request, who pays (paid_by: one of the user's own keys at no credits, the house's free allowance, or credits) and whether that payer can now (affordable), plus the account's two priorities (image_priority, clip_priority: best, balanced or cheapest) and, per task, the selected_tier its kind's priority picks. Call it before choosing a model and take the selected tier. When the priority for the kind is not saved, ask the user once, at their first request of that kind, what matters most (best quality, balanced or lowest cost, with the prices), save the answer with set_preferences and never ask again; until then an image takes the best tier and a clip (selected_tier null) waits for the answer. When a tier is not affordable, take the best affordable one and say so; a key that is out of funds (blocked) is the user's to top up, so ask. Pass task to also get the catalog recipes made for that job; when one fits, prefer it (generate_with_recipe, or start_recipe_run for a run recipe).

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "get_benchmark"
  }
}

Input Schema

NameRequiredDescriptionDefault
kindNo
taskNo

Output Schema

NameRequiredDescriptionDefault
tasksYes
accountYes
messageYes
reviewed_atYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed5 schema fields changed
    • addedOutput schema / properties / account / properties / payer_order
      Added value: +{
      +  "items": {
      +    "type": "string"
      +  },
      +  "type": "array"
      +}
    • changedOutput schema / properties / account / required
      Previous value: -[
      -  "available_credits",
      -  "free_credits",
      -  "image_priority",
      -  "clip_priority"
      -]New value: +[
      +  "available_credits",
      +  "payer_order",
      +  "free_credits",
      +  "image_priority",
      +  "clip_priority"
      +]
    • addedOutput schema / properties / tasks / items / properties / tiers / properties / best / properties / blocked
      Added value: +{
      +  "anyOf": [
      +    {
      +      "enum": [
      +        "out_of_funds"
      +      ],
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • addedOutput schema / properties / tasks / items / properties / tiers / properties / best / properties / paid_by
      Added value: +{
      +  "type": "string"
      +}
    • changedOutput schema / properties / tasks / items / properties / tiers / properties / best / required
      Previous value: -[
      -  "model_id",
      -  "label",
      -  "credits",
      -  "free_now",
      -  "affordable",
      -  "note"
      -]New value: +[
      +  "model_id",
      +  "label",
      +  "credits",
      +  "paid_by",
      +  "free_now",
      +  "affordable",
      +  "blocked",
      +  "note"
      +]
  2. Changed6 schema fields changed
    • addedOutput schema / properties / account / properties / clip_priority
      Added value: +{
      +  "anyOf": [
      +    {
      +      "enum": [
      +        "best",
      +        "balanced",
      +        "cheapest"
      +      ],
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • removedOutput schema / properties / account / properties / clip_tier
      Removed value: -{
      -  "anyOf": [
      -    {
      -      "enum": [
      -        "best",
      -        "middle",
      -        "cheapest"
      -      ],
      -      "type": "string"
      -    },
      -    {
      -      "type": "null"
      -    }
      -  ]
      -}
    • addedOutput schema / properties / account / properties / image_priority
      Added value: +{
      +  "anyOf": [
      +    {
      +      "enum": [
      +        "best",
      +        "balanced",
      +        "cheapest"
      +      ],
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • changedOutput schema / properties / account / required
      Previous value: -[
      -  "available_credits",
      -  "free_credits",
      -  "clip_tier"
      -]New value: +[
      +  "available_credits",
      +  "free_credits",
      +  "image_priority",
      +  "clip_priority"
      +]
    • addedOutput schema / properties / tasks / items / properties / selected_tier
      Added value: +{
      +  "anyOf": [
      +    {
      +      "enum": [
      +        "best",
      +        "middle",
      +        "cheapest"
      +      ],
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • changedOutput schema / properties / tasks / items / required
      Previous value: -[
      -  "id",
      -  "kind",
      -  "group",
      -  "label",
      -  "why",
      -  "tiers",
      -  "recipes"
      -]New value: +[
      +  "id",
      +  "kind",
      +  "group",
      +  "label",
      +  "why",
      +  "tiers",
      +  "selected_tier",
      +  "recipes"
      +]
  3. Changed2 schema fields changed
    • addedOutput schema / properties / tasks / items / properties / recipes
      Added value: +{
      +  "anyOf": [
      +    {
      +      "items": {
      +        "additionalProperties": false,
      +        "properties": {
      +          "action": {
      +            "type": "string"
      +          },
      +          "description": {
      +            "type": [
      +              "string",
      +              "null"
      +            ]
      +          },
      +          "id": {
      +            "type": "string"
      +          },
      +          "name": {
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "id",
      +          "name",
      +          "action",
      +          "description"
      +        ],
      +        "type": "object"
      +      },
      +      "type": "array"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • changedOutput schema / properties / tasks / items / required
      Previous value: -[
      -  "id",
      -  "kind",
      -  "group",
      -  "label",
      -  "why",
      -  "tiers"
      -]New value: +[
      +  "id",
      +  "kind",
      +  "group",
      +  "label",
      +  "why",
      +  "tiers",
      +  "recipes"
      +]
  4. Added

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only, idempotent, and non-destructive, but the description adds substantial context beyond them: how pricing, paid_by, affordable, blocked, and selected_tier null are interpreted, and the user-asking workflow when no priority is saved. That is exactly the behavioral detail an agent needs before invoking it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense and long, but it is front-loaded with the core purpose and every sentence adds operational guidance. It could be split into clearer sentences or bullets, but there is little wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations covering safety and an output schema covering return values, the description covers the remaining decision context thoroughly: pricing, payer semantics, affordability fallback, priority persistence, null selected_tier behavior, and recipe routing. Nothing critical for correct invocation appears to be missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry the semantic burden. It does so well for task, explaining that passing it returns catalog recipes for that job and should lead to generate_with_recipe or start_recipe_run, while kind is used throughout to explain priorities and selected_tier; the remaining gap is that kind's allowed values are left to the enum without a prose definition.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource and job: choosing a model across three pricing tiers, with payer, affordability, account priorities, and per-task selected tier. It distinguishes this from catalog/model-listing siblings by explaining that it is the decision tool to call before choosing a model.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says to call the tool before choosing a model and take the selected tier, and it names alternatives and follow-ups: set_preferences when a priority is not saved, generate_with_recipe or start_recipe_run when a task recipe fits. It also covers fallback behavior when a tier is unaffordable and when a key is blocked.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources