Skip to main content
Glama

📊 LLM Latency Tracker

Measure AI API latency and uptime across regions, and publish the results as open data.

🌐 Live rankings · JSON API · MCP endpoint

⚡ What it does

  • 🌍 Regional measurements — compare provider response times and availability from distributed probe nodes.

  • ⏱️ Two probe types — network time to first byte (TTFB) and optional inference time to first token (TTFT).

  • 📈 Open dataset — daily rankings, historical snapshots and regional p50 / p95 statistics.

  • 🤖 Agent access — JSON API, MCP, Markdown pages and an llms.txt index.

  • 🗓️ Model lifecycle — a deprecation calendar with links to provider announcements.

Related MCP server: Tickerr - Live AI Tool Status & API Pricing

🔧 Built for measurement

  • Python standard-library core for edge probes; provider keys are needed for inference probes.

  • SQLite storage and regional aggregation, with remote nodes shipping measurements to a central node.

  • Static publishing for the website and JSON datasets.

  • CI checks for tests and linting; published snapshots include their measurement date.

Reading the results: edge TTFB measures network/API responsiveness, not model generation speed. Compare it separately from inference TTFT; results depend on region, probe type and time window.

🚀 Quick start

Use the public data without installing anything:

curl https://llmlatency.dev/api/rankings.json

Or run edge probes locally with Python 3.12+:

git clone https://github.com/mazamaka/llm-latency-tracker.git
cd llm-latency-tracker
REGION=local python3 run.py
python3 aggregate.py --region local

For a local stdio MCP server over the published API (stdlib only, no keys):

python3 mcp_server.py

Tools: get_ai_api_latency (provider leaderboard per region), get_provider_latency (one provider across regions), get_model_deprecations (model shutdown calendar), get_coding_agent_health (Claude Code / Codex speed and accuracy drift). All read-only.

Inference probes, API access & site generation → · Deployment →

🧪 Contributing

Add a provider, bring a new probe region, or improve the measurement code. See CONTRIBUTING.md for setup and checks.


Python · SQLite · JSON API · MCP · Cloudflare Pages

Code: MIT · Data: CC-BY-4.0, with attribution to llmlatency.dev.

Daily snapshot — 2026-10-31

Measured latency across 46 AI inference providers in 4 regions. Method: distributed edge (DNS→TCP→TLS→TTFB) + inference (TTFT) probes, last 24h. License: CC-BY-4.0.

Region

Fastest provider (p50)

p50

p95

Uptime

Asia (Tokyo)

fireworks

22 ms

73 ms

100%

Europe (Germany)

fireworks

19 ms

57 ms

100%

South America (São Paulo)

openrouter

61 ms

90 ms

100%

US (Central)

fireworks

28 ms

74 ms

100%

Snapshot generated 2026-10-31T07:46:21Z — this table is regenerated daily.

Available Tools

4 tools
get_ai_api_latencyAI API latency leaderboard by regionA
Read-onlyIdempotent
Inspect

Rank hosted AI inference API providers (46 tracked: OpenAI, Anthropic, Google, Groq, Fireworks, OpenRouter and others) by measured network latency inside a probe region over the last 24 hours. Use it to answer 'which provider responds fastest from Europe/US/Tokyo/Sao Paulo' or to pick an endpoint close to your users. To follow one named provider across regions use get_provider_latency instead. Values are edge TTFB (DNS+TCP+TLS+first byte from probes every 5 minutes, direct connections, no gateway), not tokens per second or model quality. Returns the region label, dataset timestamp and the ranked rows; omit region to get all four regions at once (larger response).

NameRequiredDescriptionDefault
limitNoKeep only the top N providers per region (default: all ranked providers).
regionNoProbe location: eu-hetzner (Germany), us-central (USA), ap-tokyo (Japan), sa-east (Sao Paulo). Omit to return every region.

Output Schema

NameRequiredDescription
sourceNoAttribution URL (CC BY 4.0)
windowNoMeasurement window, e.g. last 24h
regionsYesRegion id -> {label, network: ranked rows}
generatedYesUTC timestamp when the dataset was built

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnly, idempotent, openWorld, non-destructive), the description defines what is actually measured (edge TTFB = DNS+TCP+TLS+first byte), the probe cadence (every 5 minutes, direct connections, no gateway), the response contents (region label, dataset timestamp, ranked rows), and the cost trade-off of omitting region (larger response). This is unusually rich behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and scope, and every sentence carries distinct information (use cases, sibling routing, metric definition, return shape, region omission). It is on the long side for a two-parameter tool, but no sentence is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, yet the description still summarizes them helpfully. Combined with the metric definition, region omission behavior, and sibling routing, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3; the description earns a notch above by adding the consequence of omitting region ('all four regions at once (larger response)'), which the schema omits, and by restating the human-readable region mapping. The 'limit' parameter is left to the schema, which is acceptable given full coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action (rank hosted AI inference API providers), the exact metric (measured network latency inside a probe region), the time window (last 24 hours), and the population size (46 tracked, with named examples). It explicitly distinguishes itself from sibling get_provider_latency, so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete triggering questions ('which provider responds fastest from Europe/US/Tokyo/Sao Paulo') and the decision rule to pick an endpoint near users. It names the alternative (get_provider_latency) and the condition that selects it (following one named provider across regions), and states exclusions (not tokens per second or model quality).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_coding_agent_healthClaude Code / Codex speed and accuracy driftA
Read-onlyIdempotent
Inspect

Report whether the Claude Code or Codex CLI coding agents (on consumer subscriptions) got slower, think less or answer worse than their own recent baseline. Gives output speed (tokens/s and ms per token), thinking tokens, time to first token and error rate for the last 24 h versus the previous 7 days with Mann-Whitney significance flags, plus accuracy on a fixed hard question panel. Use it when someone asks 'is Claude Code slow / nerfed today'. Measured hourly from one EU server, one account per vendor, so it reflects that vantage point; for raw API latency use get_ai_api_latency.

NameRequiredDescriptionDefault
agentNoclaude-code (Anthropic Claude Code) or codex (OpenAI Codex CLI); omit for both.

Output Schema

NameRequiredDescription
rulesNoSignificance rules used for every verdict
agentsYesPer agent: model, harness version, speed/thinking/ttft {recent, baseline, change, p, significant}, errors_24h and hard-panel accuracy
vantageNoWhere and how the agents are measured
generatedNoUTC timestamp of the export

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly/idempotent/openWorld, but the description adds non-obvious operational context: hourly measurement from a single EU server and one account per vendor, the 24h-vs-7day window with Mann-Whitney significance flags, and the fixed hard-question accuracy panel. It also discloses the vantage-point limitation, which an agent needs to interpret results honestly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense but front-loaded sentences: purpose and metrics first, then the usage trigger, then the caveat and alternative. Every sentence carries weight, though the metric list is long enough to flirt with over-specification.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering safety, the description still supplies the measurement methodology, comparison window and statistical method needed to interpret the return values correctly. Nothing material to correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single enum parameter already documents both values plus the omit-for-both behavior. The description adds no parameter syntax beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb+resource ('Report whether the Claude Code or Codex CLI coding agents got slower, think less or answer worse than their own recent baseline') and enumerates the exact metrics returned. It also explicitly separates itself from sibling get_ai_api_latency for raw API latency.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit trigger phrase ('Use it when someone asks is Claude Code slow / nerfed today') and an explicit alternative condition ('for raw API latency use get_ai_api_latency'). This is exactly the when/when-not/alternative pattern.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_model_deprecationsAI model deprecation and shutdown calendarA
Read-onlyIdempotent
Inspect

List announced shutdowns of AI models (OpenAI, Anthropic, Mistral, Azure OpenAI and others) with announcement date, shutdown date, recommended replacement and the provider page each entry was verified against, plus how many days of notice each provider typically gives (median/min/max). Use it before pinning a model version, to plan migrations, or to check whether a model ID still works. Returns upcoming shutdowns by default; set include_past to also get already-retired models. This is calendar data, not latency; for speed use get_ai_api_latency.

NameRequiredDescriptionDefault
providerNoFilter to one provider, case-insensitive: openai, anthropic, mistral, azure-openai, cohere. Omit for all providers.
include_pastNoAlso return models whose shutdown date has passed (default false).

Output Schema

NameRequiredDescription
pastNoRetired models (only with include_past)
sourceNoHuman-readable calendar page
upcomingYesModels not yet shut down, soonest first
generatedNoUTC timestamp when the calendar was built
notice_period_daysNoPer provider: models counted and median/min/max days between announcement and shutdown

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, openWorld), so the bar is lower. The description adds genuinely useful behavior beyond them: default scope is upcoming shutdowns only, include_past widens it, and entries carry a provider page they were verified against. It stops short of describing caching, freshness, or rate limits, so not a full 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with what the tool returns, then usage, then the sibling disambiguation. Every clause carries information, though the long returned-field enumeration is dense enough that it approaches a data dump.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only lookup with a full output schema and complete annotation coverage, nothing an agent needs is missing: scope defaults, filter behavior, replacement guidance, and sibling routing are all present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented with valid provider values and the include_past default. The description restates the include_past effect ('also get already-retired models') and omits provider filter syntax, adding little beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (list) and resource (announced AI model shutdowns), enumerates the fields returned (announcement date, shutdown date, replacement, verification page, notice statistics), and explicitly distinguishes itself from the sibling latency tools. An agent can identify this tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use triggers ('before pinning a model version, to plan migrations, or to check whether a model ID still works') and names the alternative for a different need ('for speed use get_ai_api_latency'). Both the selection condition and the exclusion are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_provider_latencyOne AI provider's latency in every regionA
Read-onlyIdempotent
Inspect

Show how a single AI API provider performs from every probe region (Germany, USA, Tokyo, Sao Paulo): its rank in that region, median and p95 time to first byte, uptime and sample count over the last 24 hours. Use it for questions like 'is the Anthropic API slower from Asia than from Europe' or 'was OpenAI reachable today'. For the full leaderboard of a region use get_ai_api_latency. An unknown provider id returns an error that lists the valid ids.

NameRequiredDescriptionDefault
providerYesProvider id as published on llmlatency.dev, lowercase with hyphens, e.g. openai, anthropic, google, groq, mistral, deepseek, openrouter, together, fireworks.

Output Schema

NameRequiredDescription
sourceNoAttribution URL (CC BY 4.0)
regionsYesOne row per region where the provider was measured
providerYesNormalized provider id
generatedNoUTC timestamp when the dataset was built

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, openWorldHint and destructiveHint=false, so safety is covered structurally. The description adds meaningful behavior beyond them: the 24-hour data window, the enumerated probe regions, the exact metric set, and the error contract for an unknown provider id. Return format is not detailed, but an output schema exists to carry that.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with what the tool returns, then examples, then the alternative tool. No filler, no restatement of the title, and each sentence carries distinct routing or scoping information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values needn't be explained, yet the description still names the metrics and regions the agent will see. Combined with usage examples, the sibling routing rule, and the error behavior, everything needed to select and call this single-parameter tool correctly is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the format is already pinned by the schema (lowercase-with-hyphens, example ids), so the baseline is 3. The description adds a semantic the schema does not: an unknown provider id errors and lists the valid ids, which tells the agent how to recover from a bad value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource: 'Show how a single AI API provider performs from every probe region', with the exact metrics returned (rank, median and p95 TTFB, uptime, sample count) and the time window (24 hours). It explicitly contrasts scope with the sibling get_ai_api_latency, which is the full leaderboard, so an agent can distinguish them without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides concrete example questions ('is the Anthropic API slower from Asia than from Europe', 'was OpenAI reachable today') that map intent to this tool, and names the alternative get_ai_api_latency plus the condition that selects it (full region leaderboard). When-to-use and when-not-to-use are both covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observedget_ai_api_latency
    • First observedget_coding_agent_health
    • First observedget_model_deprecations
    • First observedget_provider_latency

TDQS

A4.5/5.0

Scored across 4 tools

Disambiguation5/5

The two latency tools have overlapping subject matter, but each description explicitly points to the other with a clear boundary rule (leaderboard vs. single-provider across regions). The deprecations and coding-agent-health tools are in entirely different domains, and each description reiterates its scope ('calendar data, not latency', 'for raw API latency use...'). An agent can pick correctly without guessing.

Naming Consistency5/5

All four tools use the same get_<subject> snake_case pattern with no deviation in verb or casing. The names are predictable and readability is high. Minor quibble: get_ai_api_latency actually returns a ranked leaderboard rather than a single latency value, but the convention itself is uniform.

Tool Count4/5

Four tools is compact and none feel redundant, which fits a narrowly scoped observability server. It is on the lean side given it spans three distinct areas (API latency, model deprecations, coding-agent health), leaving little room for supporting lookups.

Completeness4/5

The surface covers the core questions: regional ranking, per-provider detail, deprecation planning, and agent regression checks. Gaps are minor and workaroundable (no provider-list endpoint, though errors enumerate valid ids; no historical time series beyond the fixed 24h window).

Maintenance

ActivityActive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    D
    maintenance
    Real-time status monitoring, uptime tracking, incident history, and API pricing for 42+ AI tools including ChatGPT, Claude, Gemini, Cursor, GitHub Copilot, Perplexity, DeepSeek, and Groq. No API key required. Data updated every 5 minutes from independent monitoring infrastructure.
    7
    61 npm
    1
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Exposes queryable GPU inference benchmark data (quantization, throughput, VRAM, concurrent users) as tools for LLM clients.
    MIT