List all 21 providers with their status (configured/unconfigured, enabled/disabled, model count).

Call this first to see which providers you have API keys for. No network requests — instant response.

List models in the catalog without pinging. Use when the user asks what models are available or to browse by tier/provider/free.

Response includes model_id (the code name to use when inserting/configuring, e.g. run(prompt, model_id=...) or Cursor settings) and label (display only).

Args: tier: Filter to exact tier (S+, S, A+, A, A-, B+, B, C) provider: Filter to provider key (nvidia, groq, cerebras, etc.) min_tier: Show this tier and above (e.g. "A" = A, A+, S, S+) free_only: If true, only list models marked as free (from API or :free/-free in id)

Ping models in parallel and return ranked results by latency. Use when you need live speed data or a ranked list.

Pings all matching models, returns sorted fastest-first. Takes 2-10 seconds depending on filters.

When verify=True, sends a real prompt to each "up" model and checks for non-empty content. Models that return empty/garbage are marked as BROKEN (distinct from ERROR or OVERLOADED). This catches models that ping as UP but are functionally dead. Verification results are cached across scans within the session.

Args: tier: Filter to exact tier (S+, S, A+, A, A-, B+, B, C) provider: Filter to provider key (nvidia, groq, cerebras, etc.) min_tier: Show this tier and above (e.g. "S" shows only S+ and S) configured_only: Only ping models whose provider has an API key free_only: Only include models marked as free (from API or :free/-free in id) limit: Max results (default 20, 0 = all) verify: Send a real prompt to validate non-empty content (default false) verify_prompt: Custom verification prompt (default "Reply with exactly: OK")

Get the N fastest available models right now. Use when the user wants recommendations or "best/fastest/free" models.

Pings configured providers and returns top N by latency. Use model_id from results as the code name when inserting or configuring (e.g. run(prompt, model_id=..., provider=...)). Example: get_fastest(free_only=True, min_tier="A", count=5) for "5 free A-or-better models".

When verified=True, also sends a real prompt to each model to confirm it produces non-empty output. Models that ping as UP but return empty content are excluded.

Args: min_tier: Minimum quality tier (default "A" — shows S+, S, A+, A) provider: Limit to specific provider count: How many results (default 5) free_only: If true, only return models marked as free verified: If true, verify models produce non-empty output (default false)

Check health of all configured providers by pinging one model from each.

Returns per-provider latency and status. Useful for diagnosing which providers are currently responsive vs overloaded.

Set an API key for a provider. Saved to ~/.model-radar/config.json.

Args: provider: Provider key (nvidia, groq, cerebras, sambanova, openrouter, huggingface, replicate, deepinfra, fireworks, codestral, hyperbolic, scaleway, googleai, siliconflow, together, cloudflare, perplexity) api_key: The API key value

Fetch latest model lists from configured providers (openrouter, nvidia, groq) and replace them in the database.

Only providers with API keys are fetched; their previous model list is discarded and replaced with the live API list. Other providers keep their existing list. Use this to get the current catalog, then call scan() or get_fastest() for discovery. Optionally run a quick ping test after refresh.

Args: provider: Optional provider to refresh only (openrouter, nvidia, groq) run_ping: If true, run a ping test on up to ping_limit models after refreshing ping_limit: Max models to ping when run_ping is true (default 20)

Run a prompt on the fastest available model and return the response.

Use when the user wants to execute a prompt. Picks the fastest responding model automatically (with optional fallback). Set free_only=True when the user asks for a free model only.

Args: prompt: The user message to send system_prompt: Optional system prompt (e.g. "You are a Python expert") model_id: Specific model to use (skips scanning). Use list_models() to browse. provider: Limit to a specific provider (nvidia, groq, etc.) min_tier: Minimum quality tier when auto-selecting (default "A") free_only: If true, only consider models marked as free (default false) max_tokens: Max response tokens (default 4096) temperature: Sampling temperature (default 0.0 for deterministic)

Quality-test models with 5 coding challenges and return pass/fail scores.

Runs arithmetic, instruction following, code generation, code reasoning, and JSON output challenges. Catches models that are fast but hallucinate, ignore instructions, or produce garbled output.

Without model_id, scans for the fastest models and benchmarks the top N.

Args: model_id: Specific model to benchmark (optional) provider: Limit to a specific provider (nvidia, groq, etc.) min_tier: Minimum quality tier when auto-selecting (default "A") count: How many models to benchmark when auto-selecting (default 3)

Run the same prompt on multiple models in parallel and return all responses.

Use this for verification and consensus. When accuracy matters more than speed, ask N models the same question and compare their answers. If 3/3 models agree, you can be more confident in the result.

Returns all responses side-by-side with model info, latency, and quality scores (if previously benchmarked).

Args: prompt: The question or task to send to all models system_prompt: Optional system prompt applied to all models count: How many models to query in parallel (default 3) min_tier: Minimum quality tier for model selection (default "A") provider: Limit to a specific provider (nvidia, groq, etc.) max_tokens: Max response tokens per model (default 4096) temperature: Sampling temperature (default 0.0 for deterministic)

Run multiple prompts in parallel with bounded concurrency and auto-retry.

For batch workloads: translation pipelines, data extraction, classification, content generation. Picks the fastest model and runs all prompts through it. Failed items are automatically retried on alternate models.

Each prompt dict should have a "prompt" key and optional "system_prompt" (overrides the top-level system_prompt) and "metadata" keys for tracking.

When results_file is set, each completed item is appended as a JSON line immediately. If interrupted, the file contains all completed items and can be resumed (already-completed indices are skipped).

Args: prompts: List of {"prompt": "...", "system_prompt": "...", "metadata": {...}} system_prompt: Default system prompt for all items (per-item overrides) model_id: Specific model to use (skips scanning). Use list_models() to browse. provider: Limit to a specific provider min_tier: Minimum quality tier when auto-selecting (default "A") free_only: If true, only use free models max_tokens: Max response tokens per item (default 4096) temperature: Sampling temperature (default 0.0) concurrency: Max parallel requests (default 5) retry_on_fail: Auto-retry failed items on alternate models (default true) results_file: Path to JSONL file for incremental writes and resume support

Rate a single item using N diverse judge models and return aggregate scores.

Auto-selects judges spread across different providers for independence. Enforces structured output (CSV or JSON scores), retries on malformed responses, and computes inter-rater agreement metrics.

Use this for evaluation tasks: rating translations, code quality, content accuracy, or any rubric-based assessment.

Args: prompt: The evaluation prompt (describe what to rate and provide the content) rubric: List of scoring dimensions (e.g. ["accuracy", "naturalness", "completeness"]) scale: Rating scale as "min-max" (default "1-5", also supports "1-10") count: Number of judge models to use (default 3) min_tier: Minimum quality tier for judge selection (default "A") free_only: If true, only use free models as judges output_format: How judges format scores — "csv" (default) or "json" max_tokens: Max response tokens per judge (default 256) temperature: Sampling temperature (default 0.0)

Blind A/B comparison of two items judged by N models.

When blind=True (default), randomizes which item is shown as A vs B to each judge independently, then de-randomizes scores. This prevents position bias where judges consistently favor the first item shown.

Use for comparing translations, code solutions, summaries, or any pair of outputs where you want an objective preference.

Args: item_a: First item to compare item_b: Second item to compare context: Optional context for the comparison (e.g. the original task) dimensions: Scoring dimensions (default ["quality"]) scale: Rating scale as "min-max" (default "1-5") judge_count: Number of judge models (default 3) blind: Randomize A/B order per judge to prevent position bias (default true) min_tier: Minimum quality tier for judge selection (default "A") free_only: If true, only use free models as judges max_tokens: Max response tokens per judge (default 512) temperature: Sampling temperature (default 0.0)

Run judge evaluations at scale on a list of items.

Processes items with bounded concurrency using a shared pool of diverse judges. Returns per-item scores, summary statistics (mean, stdev, min, max per dimension), and error counts.

Each item in the list should have a "prompt" key with the evaluation prompt, and an optional "metadata" key for tracking (e.g. language, entry ID).

When results_file is set, each scored item is appended as a JSON line immediately after scoring. On interruption, the file contains all completed items. On resume (same results_file), already-scored indices are skipped automatically.

Args: items: List of {"prompt": "...", "metadata": {...}} dicts rubric: List of scoring dimensions (e.g. ["accuracy", "naturalness"]) scale: Rating scale as "min-max" (default "1-5") judge_count: Judges per item (default 3) min_tier: Minimum quality tier for judge selection (default "A") free_only: If true, only use free models as judges output_format: How judges format scores — "csv" (default) or "json" concurrency: Max items evaluated in parallel (default 5) max_tokens: Max response tokens per judge (default 256) temperature: Sampling temperature (default 0.0) results_file: Path to JSONL file for incremental writes and resume support

Get N verified-alive models across N distinct providers, ranked by tier then latency.

The single most common pattern for translation pipelines and batch evaluation: "give me N working models from N different providers". Returns model_ids ready to use with run(model_id=...) or batch_run(model_id=...).

Provider diversity is enforced: at most 1 model per provider. Models are verified alive by default (sends a real prompt to confirm non-empty output).

Args: count: Number of workers to return (default 5) min_tier: Minimum quality tier (default "A") free_only: Only include free models (default false) verified: Verify models produce non-empty output (default true)

Evaluate a translation via back-translation and gloss overlap.

Translates the output back to the source language using a different model, then computes word-level gloss overlap with the original text. This is the most powerful non-circular quality metric for translation: original → translate → back-translate → overlap.

Returns the back-translation, overlap score (0.0-1.0), matching/missing/extra glosses, and the model used for back-translation.

Args: text: Original source-language text (e.g. "father, head of household") translation: The translated text to evaluate (e.g. "Vater, Haupt eines Haushalts") source_lang: Source language name (e.g. "English") target_lang: Target language name (e.g. "German") back_model_id: Specific model for back-translation (default: auto-select different model) min_tier: Minimum quality tier for auto-selection (default "A") free_only: Only use free models (default false) max_tokens: Max response tokens (default 512)

Get setup instructions for adding free model providers.

Without arguments, returns a prioritized list of all unconfigured providers with signup URLs, free tier details, and setup steps. With a provider argument, returns detailed instructions for that specific provider.

Use this to help your user expand their model coverage. More providers means better fallback options and more models to choose from.

Args: provider: Specific provider key to get instructions for (optional). Omit to see all unconfigured providers.

Deterministic setup workflow: guide the host to get API keys installed.

Run steps in order. Step 1: check/install Playwright (optional). Step 2: get list of remaining (unconfigured) providers — then prompt the user to choose which to set up. Step 3: pass that selection as provider_selection to get login instructions and where to save each key. Step 4: summary of where keys are stored (config path, configure_key tool, env vars).

Many providers support GitHub SSO; the response marks them so the host can tell the user they may only need to click "Sign in with GitHub" and allow.

Args: step: 1 (Playwright), 2 (remaining providers), 3 (login + save), 4 (where to save). provider_selection: For step 3 only. List of provider keys (e.g. groq, openrouter) the user chose from step 2. If omitted at step 3, response tells you to prompt the user and call again with selection.

Tell the host agent where to search the machine to swap in a model-radar model.

Returns: (1) Where model-radar stores API keys. (2) OpenAI-compatible base_url and model_id for the given model (or a recommended min_tier model). (3) Per-app search locations for Cursor, Claude Code, Open Interpreter, OpenClaw — with paths for Linux, Mac, Windows, and WSL (e.g. ~/.cursor, /mnt/c/Users//.cursor). The host can search these paths and set base_url + model_id + API key so the app uses a free model from model-radar.

Args: model_id: Specific model_id (e.g. llama-3.3-70b-versatile). Omit to get a recommended model at min_tier. provider: Limit to this provider when choosing a recommended model. min_tier: When model_id is omitted, recommend a model at this tier or better (default A).

Request the server to exit so a process manager can restart it (SSE only).

When running model-radar with SSE, a process manager or wrapper can restart the server on exit. This tool exits the process with code 0 so the manager starts a fresh process (and loads any updated code/tools). The client must reconnect after the restart. Restart is allowed by default; set MODEL_RADAR_ALLOW_RESTART=0 to disable.

Return when this server process started and how long it has been running.

Use to answer questions like 'how fast did model-radar come up' or 'how long has the server been running'. started_at is set when the server process began (create_server was called); uptime_seconds is seconds since then.