Skip to main content
Glama

Quick Start

You need Ollama running with at least one model pulled (ollama pull llama3.2). Then:

npx llmxray

Or with Docker:

docker run -p 5174:5174 -p 11435:11435 djovaneli/llmxray

Now point your tools at http://localhost:11435 instead of Ollama's 11434: VS Code Copilot, Open WebUI, Continue, n8n, your own scripts. It behaves exactly like Ollama and forwards everything. Open http://localhost:5174, go to Live, and watch their turns arrive.

Just want to try a model? Open the Chat page, pick one, and send a message.


Related MCP server: devenv-doctor-mcp

What your tools really send

Your editor, your agents and your scripts all talk to the same Ollama. These pages show what each of them sends, what the daemon does with it, and what that costs the others.

Live: what your other tools send

LLMxRay opens a second port, 11435, that behaves exactly like Ollama and forwards everything to it. Point VS Code Copilot, Open WebUI, Continue, n8n or your own script at http://localhost:11435 and every turn they send appears here: client, model, prompt size, how much came from the cache, prefill and generation speed, tool calls.

VS Code Copilot ─┐
Open WebUI ──────┤                                    ┌─ Live page: every turn,
Continue / n8n ──┼─► http://localhost:11435 ─► Ollama ┤  cache hits, prefill speed,
your scripts ────┘        (LLMxRay)                   └─ where the cache broke

Live: turns from two clients, with findings

Select a turn to see what the prompt is made of and the exact point where it stopped matching the previous one, which is where the cache broke. Findings are plain sentences, each linked to the turn that proves it: a client re-reading its whole prompt every turn, a prompt the daemon silently cut to fit the context (a chat that outgrew it loses its oldest messages; instructions too long on their own lose their start; Ollama says nothing either way), a model running partly on CPU, tool schemas taking 40% of every prompt. Only counts and timings are kept, on this machine for 30 days, so a restart picks up where it stopped. Prompt text is an opt-in that writes to a local file. Nothing leaves your machine unless you send the turns on to your own trace viewer: each model call as an OpenTelemetry span, each run as a trace, for Jaeger, Phoenix, Langfuse or Grafana.

Both ports answer only this machine by default, because the Live log holds whatever your tools sent. Start with npx llmxray --host 0.0.0.0 to let other machines on your network reach them, for a Live page on a second screen or a shared GPU box.

Prompt X-ray: the prompt itself, drawn to scale

Each turn your tool sent, as a bar. Colour is what each part of the prompt was: system prompt, tool schemas, earlier turns, the new message. Brightness is what happened to it: dim was already in the model's memory and cost nothing, bright had to be read again, striped was cut by the daemon and never seen. Stack the turns of a conversation and you watch the cache grow, see the turn where a timestamp in the system prompt emptied it, and the turn where the conversation outgrew the context and Ollama silently dropped the oldest messages and answered anyway. When the system prompt alone is too long, it is its start that goes. Click a part and the request opens at that place in the text.

Prompt X-ray: each turn of a real agent run drawn to scale, cached parts dim, recomputed parts bright

Runs: one tool's work, end to end

One tool's calls to the model laid out in time, as a waterfall. Each call is split into what Ollama reports: waiting, loading the model, reading the prompt, writing the answer. Between calls, the tool's own time, named after the tools the model asked for. Steps start where the tool began a fresh conversation, so an agent that throws its cache away at every step shows it at a glance, and so does a model that was not in memory when the run began. Click any row for what that call sent and what came back, the model's thinking and its tool calls with their arguments included, or for what a tool was asked and what it returned. Every row has its own link. Switch on Keep the prompt text on the Live page to see the text; the timings need nothing.

Runs: a 36-call agent run as a waterfall, with a tool's input and result

Shared box: why Ollama is slow right now, and which tool did it

One line per tool, sorted by what it cost the others, in seconds: the one that sends keep_alive: 0 and unloads the model after every run, the one that asks for a 128K context on 4K prompts and drops the daemon to one request at a time, the one that waited 22 seconds behind them. Each with the setting to change. A tool that cost nothing is listed too, so you can see what good looks like.

Shared box: what one tool asked of the daemon and what it cost the others

GPU map: what is in video memory, and what else would fit

One bar the width of your card: each loaded model as a block split into its weights and what its context costs on top, then the free space. Under it, every pulled model with one of three verdicts: fits now, fits only if what is loaded unloads first, larger than the card.

GPU map: a loaded model at its real size, with what else would fit

Tool calls: can this model drive an agent?

Thirteen request shapes agents send all day, scored on what the daemon returned: the right tool, the right arguments, the right number of calls, no call when none is needed. Each cell is one sentence: "the path argument is missing", "wrote the call into the answer text", "called search_web when no tool was needed". Click it for what the model returned.

Tool calls: thirteen agent request shapes scored on gemma4

Cache Lab

A local model reuses its prompt cache only while the prompt matches from the very first token. One timestamp near the top forfeits everything below it. Cache Lab finds the values that change between turns, shows where reuse dies, and then measures against your own daemon what moving them to the end saves.

Cache Lab: diagnosis and measured result

Measured on a real 324-token prompt: 4 tokens reused and 64.6 ms of prefill with the timestamp at the front, 290 reused and 18.6 ms with it at the back. 3.5x faster, same words.

Protocol Observatory

The same prompt through Ollama's three serving protocols in parallel: native /api/chat, OpenAI-compatible /v1/chat/completions, and Anthropic-compatible /v1/messages. Side-by-side streaming, per-protocol metrics, and an envelope diff that shows how each one frames finish reasons, token counts and errors.

Protocol Observatory: three protocols side by side


Keep it, and send it on

Every turn's record (tokens, cache, timings, client, model, never the text) stays on your machine for 30 days, so a restart picks up where it stopped. Live, Runs, Prompt X-ray and the MCP tools carry on as if nothing happened.

If your team already reads traces, send the turns there: each model call becomes an OpenTelemetry span with the GenAI attributes viewers already read, each run a trace, and the tool time between calls its own span, the same waterfall as Runs. Set an address on the Live page, or OTEL_EXPORTER_OTLP_ENDPOINT.

Viewer

Address

Jaeger (docker run -p 16686:16686 -p 4318:4318 jaegertracing/jaeger)

http://localhost:4318

Arize Phoenix (docker run -p 6006:6006 arizephoenix/phoenix)

http://localhost:6006

Langfuse

https://cloud.langfuse.com/api/public/otel, with a Basic header from your keys

Grafana Tempo, an OpenTelemetry Collector

its OTLP HTTP address, usually port 4318

The prompt text goes along only if you switch on both Keep the prompt text and Include the prompt text in each span.


For agents

If you are an agent running on a local model, LLMxRay can tell you what went wrong with your own calls. Two steps.

  1. Send your Ollama calls to port 11435 instead of 11434. It behaves exactly like Ollama and forwards everything; LLMxRay only records what passes.

  2. Add the LLMxRay MCP server to your tools:

{ "mcpServers": { "llmxray": { "command": "npx", "args": ["-y", "-p", "llmxray", "llmxray-mcp"] } } }

Seven tools, all read-only, nothing leaves the machine:

Tool

Answers

diagnose_my_turns

What stands out in your recent turns, as sentences with the fix: a prompt rejected for exceeding the loaded context, the prompt cache lost on every turn or on one turn, a model running partly on CPU, tool schemas crowding the prompt.

read_my_last_turn

Your last turn in numbers: prompt tokens and cached share, prefill time and speed, generated tokens and speed, why generation stopped, what the prompt was made of, whether the model fit in video memory.

list_local_models

Which models are pulled, their size and quantisation, which are loaded now and with what context, and whether each fits in video memory.

list_my_turns

The history: one line per turn with tokens, cached share, prefill, generation speed, stop reason and flags.

compare_my_turns

Did a change help: medians of cached share, prefill, speed and prompt size before and after a moment, with a verdict.

estimate_my_prompt

Before sending: does a prompt of N tokens fit the loaded context, how much headroom is left, does the model sit in video memory, how much the cache would reuse.

read_llmxray_status

Where LLMxRay runs, the observe port, the Ollama it forwards to, how many turns it keeps.

LLMxRay itself must be running (npx llmxray). The MCP server reads from it over localhost.

The same procedure, as an Agent Skill your agent can load: npx skills add LogneBudo/llmxray (skills.sh) or clawhub install llmxray-diagnose-ollama-calls (ClawHub). Source: skills/diagnose-ollama-calls/SKILL.md.

Name yourself so your turns are yours: send the header X-LLMxRay-Client: my-agent, or use http://localhost:11435/?client=my-agent as the base URL, and pass my-agent as client to the tools. Where to set the base URL: Ollama SDKs and CLI OLLAMA_HOST=127.0.0.1:11435; OpenAI-compatible clients base_url=http://localhost:11435/v1; Claude Code ANTHROPIC_BASE_URL=http://localhost:11435; Continue apiBase; Open WebUI OLLAMA_BASE_URL.


Also in the box

You run a local model. You chat with it. LLMxRay answers the questions that come next: how fast was each token, what did the prompt cost, did the cache help, which setting was actually better, and what are your other tools sending to the same model.

Every page below is one of those answers.

Chat, token by token

Every token is drawn as it arrives, coloured by the model's own probability for it, from Ollama's logprobs. You see where the model hesitated and where it was sure. Markdown, multi-turn, file and image attachments, slash commands, and a thinking budget for reasoning models (off, model's choice, or low / medium / high / max).

Each response is also checked as it lands. A badge appears only when something is wrong: repetition, a refusal, gibberish, an empty answer, or a reply cut off at the token limit.

Chat: a response streaming, each token coloured by its probability

Session deep dive

Open any turn to see it as a timeline: time to first token, inter-token gaps, the reasoning block if there was one, tool calls, and the exact prompt that was sent.

Session details: stream timeline and metrics

Compare

The same prompt against up to four slots, each with its own model, temperature and system prompt, streaming side by side. One-click presets: Temperature Sweep, Deterministic Pair, Language Compare. A word-level diff shows exactly what changed between two answers.

Compare: temperature sweep on one model

Surgical Benchmark

Score models on curated question suites (MMLU-Pro, ARC-Challenge, HellaSwag, GSM8K, TruthfulQA) using the model's real token logprobs, not whether the answer happened to look right. Confidence versus accuracy per category, so you see where a model is confidently wrong. Import your own suites as JSON.

Benchmark: confidence versus accuracy

Decision Lab

Decision models (nimble, tev1, clef) answer typed questions with probabilities instead of prose. Give the lab a situation and ask for a choice among candidates, a yes/no probability, or a score on a scale. Every answer comes back as the model's own distribution, drawn as bars.

Decision Lab: probability bars behind each answer

Knowledge Base and Embeddings

Drop in PDF, DOCX, Markdown or CSV files. They are chunked, embedded and searchable, so you see what retrieval actually returns before a model ever sees it. The Embeddings page lets you compare any two texts by cosine similarity, visualise the vectors, and see what Matryoshka truncation costs.

Everything stays in your browser's IndexedDB.

Knowledge Base: search results with scores

Embeddings: cosine similarity between two texts

Code Completion

Fill-in-the-middle for Qwen-Coder, CodeLlama, Codestral, DeepSeek-Coder and StarCoder: a prefix, a suffix, and the model fills the gap. The stitched preview shows the result as your editor would.

Code Completion: prefix, suffix and stitched preview

Tool Workshop

A node canvas for building tool definitions, with the TypeScript kept in sync both ways: edit the nodes or edit the code. Probe an API, generate the schema, and test the tool with a live call.

Tool Workshop: node canvas with the code panel open

Analytics and Costs

Latency percentiles (P50 / P95 / P99) for requests and first tokens, quality over turns, errors by model, a usage heatmap, and temperature against tokens per second. The cost page prices your local usage at cloud rates, so you see what running locally saves.

Analytics: latency percentiles and quality

Costs: cloud-equivalent pricing per model

My System and Models

Hardware, runtime and inference overview on one page: CPU, memory, GPU, which models are loaded and where. The Models tab lists every installed model with its parameters, template and runner, and pulls new ones from the catalog.

My System: hardware and Ollama status

Models: installed models and catalog

More

  • AI Training Pipeline. Curate pairs from your conversations, tag and review them, export JSONL for fine-tuning.

  • Local history database. Every chat, comparison and benchmark is archived in IndexedDB with filters, trends, exports and retention rules.

  • Educators Kit. Nine lesson modules on the docs site, each built around an LLMxRay page.

  • Five languages. English, French, Serbian (Latin and Cyrillic), Chinese and Arabic, with RTL layout. Scaffolds ready for Hebrew and Japanese.


Who is this for

You are...

LLMxRay helps you...

Developer

Debug prompts, profile latency, compare models, inspect tool calls, see what your editor sends

Researcher

Run controlled experiments with the same settings across models and temperatures

Student / Educator

Watch model behaviour instead of reading about it, with the nine-module Educators Kit

AI team lead

Give every developer the same local setup, and collect the benchmark results and training pairs they export


Install

npx llmxray
npx llmxray --port 3000
npx llmxray --ollama-url http://192.168.1.50:11434
npx llmxray --observe-port 11436     # the Live port, 11435 by default

Docker

docker run -p 5174:5174 -p 11435:11435 djovaneli/llmxray
docker run -p 5174:5174 -p 11435:11435 -e OLLAMA_URL=http://host.docker.internal:11434 djovaneli/llmxray

From source

git clone https://github.com/LogneBudo/llmxray.git
cd llmxray
npm install
npm run dev     # http://localhost:5173

Ollama compatibility

Tested against Ollama 0.40.x (verified on 0.40.10, October 2026). Works with 0.20 and newer; older versions chat but lack think and JSON-schema format. Recommended: 0.33.3 or newer, which reports prompt-cache reuse, so prefill speed is measured over the tokens actually evaluated.

Endpoint

Used for

/api/chat

Streaming chat (NDJSON, with tools, think effort levels, format schema)

/api/generate

Generation and Fill-in-the-Middle via suffix

/api/tags, /api/show

Model list, capabilities, context length, parameters, template

/api/embed

Vector embeddings for the Knowledge Base, with optional dimensions truncation

/api/pull, /api/delete, /api/ps, /api/version

Model management and status

/v1/chat/completions

OpenAI-compatible path, used by Surgical Benchmark for real logprobs

/v1/messages

Anthropic-compatible path, used by Protocol Observatory

/v1/systemone

Decision models, used by Decision Lab


Under the hood

Vue 3.5, TypeScript 6 (strict), Vite 8, Tailwind CSS 4, Pinia stores per concern, Chart.js and D3 for charts, Vue Flow for the tool canvas, CodeMirror 6 for code. The app keeps its own data (chats, comparisons, benchmarks, the knowledge base) in the browser's IndexedDB; the Live log's turn records live in ~/.llmxray on the machine that runs LLMxRay. The backend is Ollama, local.

Streaming reads Ollama's NDJSON through fetch() and a ReadableStream, and tokens update the UI through Pinia. Chat and Compare colour tokens by the real logprobs Ollama returns; without them a token is left uncoloured, and the token view names timing as pace, never as confidence. Benchmarks use real logprobs too. Hardware detection is a small Vite plugin that asks the OS directly. The Live port is a transparent proxy: it forwards every request to Ollama untouched, translates nothing (Ollama itself serves the OpenAI- and Anthropic-compatible routes), and records counts and timings, kept for 30 days.

Command

What it does

npm run dev

Dev server (port 5173)

npm run build

Type-check and production build

npm run test

Unit tests (Vitest)

npm run test:e2e

End-to-end (Playwright)


Contributing

Contributions welcome. See CONTRIBUTING.md for setup and guidelines. Community translations are especially welcome: scaffold files are ready for Hebrew and Japanese.


License

Apache License 2.0


Trademark

LLMxRay is a trademark of Ivan Stankovic (LogneBudo). See TRADEMARK.md.


Available Tools

7 tools
compare_my_turnsCompare my turnsA
Read-onlyIdempotent

Did a change help? Medians of cached share, prefill time, generation speed and prompt size for the turns before and after a moment, with the difference and a one-line verdict on the cache. Four tools read your turns, one question each: diagnose_my_turns, is anything wrong (a verdict over several turns); read_my_last_turn, what exactly happened on the latest turn (one turn in full); list_my_turns, what happened turn by turn (the sequence, one line each); compare_my_turns, did a change help (before against after). Use it after you changed a system prompt, keep_alive, num_ctx or the tool list. split_minutes_ago: 1 to 10080, default 60: turns older than that are "before", newer are "after"; set it to the minutes since your change. client: your X-LLMxRay-Client name, so another tool's turns do not mix in. Both sides need at least one turn, else it says what to move. Needs LLMxRay running (npx llmxray). Under a second. No model is called.

NameRequiredDescriptionDefault
clientNoSubstring of the client label, case-insensitive. Omit for all clients.
split_minutes_agoNoMinutes ago that separate before from after, 1 to 10080. Default 60.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so safety is covered. The description adds real context beyond that: it requires LLMxRay running via npx llmxray, runs under a second, calls no model, and fails with guidance ('it says what to move') when either side has no turns. Return-format detail is left to the medians/verdict summary, so it is short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The value proposition is front-loaded in one question ('Did a change help?'), followed by the metrics and the sibling routing table, then parameters and prerequisites. Nothing is redundant, though the four-tool enumeration is a fairly long detour for a single tool and slightly dilutes the core instruction.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still conveys what is returned (medians per metric, the delta, and a one-line cache verdict). It covers prerequisites, timing, parameter meaning, and the failure mode of insufficient turns on either side, which is everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description goes beyond it: split_minutes_ago is explained semantically (older = 'before', newer = 'after', set it to minutes since your change) rather than just bounded, and client is given a purpose ('so another tool's turns do not mix in'). The schema's own note that client matches a substring is not restated, so it is not a full 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a concrete verb and resource (compare turns before vs after a moment) and enumerates exactly what is computed: medians of cached share, prefill time, generation speed and prompt size, plus the difference and a one-line verdict. It explicitly positions itself against the three sibling turn-readers, so the agent can distinguish it without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the triggering situation directly: 'Use it after you changed a system prompt, keep_alive, num_ctx or the tool list.' It also maps the entire sibling family to one question each (diagnose_my_turns = verdict, read_my_last_turn = one turn in full, list_my_turns = sequence), which is an explicit alternative-selection guide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

diagnose_my_turnsDiagnose my turnsA
Read-onlyIdempotent

Is anything wrong with the Ollama calls your client sent through LLMxRay on port 11435? A verdict judged over several turns: a prompt rejected or silently cut by the context, the cache lost on every turn or on one turn, a model partly on CPU, tool schemas crowding the prompt. Four tools read your turns, one question each: diagnose_my_turns, is anything wrong (a verdict over several turns); read_my_last_turn, what exactly happened on the latest turn (one turn in full); list_my_turns, what happened turn by turn (the sequence, one line each); compare_my_turns, did a change help (before against after). Call it after a few turns, and again after a change. For which model or context fits use list_local_models. Returns text. First one line per client seen, then one finding per line tagged [high], [medium] or [low], each ending with the fix. It says so when nothing stands out, and explains how to route through 11435 when no turn was seen. client: your X-LLMxRay-Client name; without it on a shared machine you read every tool's turns. since_minutes: 1 to 10080, minutes since your change. They combine. Needs LLMxRay running (npx llmxray) and says so if not. Under a second, from records already kept. No model is called.

NameRequiredDescriptionDefault
clientNoPart of the client label or id as LLMxRay shows it (for example "VS Code Copilot", "python-requests", or the X-LLMxRay-Client header you sent). Case-insensitive substring. Omit for all clients.
since_minutesNoOnly turns from the last N minutes. Omit for every turn LLMxRay has kept.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive, closed-world, so the bar is lower, yet the description adds real behavioral detail: how findings are formatted (one line per client, then tagged [high]/[medium]/[low] each ending with the fix), that it declares when nothing stands out, that it explains 11435 routing when no turn was seen, plus prerequisites, latency, and 'no model is called'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core question and no filler sentences, but the tool-comparison block and return-format block make it longer than strictly needed; each sentence still carries information an agent would otherwise have to infer.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description compensates by describing the return text shape, the empty-case behavior, and the failure/routing path when LLMxRay is not running. Nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3; the description adds meaning beyond it by noting that client and since_minutes combine and by flagging the shared-machine consequence of omitting client (you read every tool's turns). It does not restate the substring semantics, which the schema already covers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (a verdict over several turns about Ollama calls sent through LLMxRay on port 11435) and explicitly positions itself against all four sibling turn-readers with a one-line differentiator each. An agent can pick this tool without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit timing guidance ('Call it after a few turns, and again after a change') and routes the agent to list_local_models for a different question (which model or context fits). The four-tool comparison table makes when-to-use vs alternatives unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

estimate_my_promptEstimate my promptA
Read-onlyIdempotent

Before sending a prompt of N tokens: whether it fits the context the model is loaded with and the headroom left for the answer. Also whether the model sits in video memory, and how much of the prompt the cache would reuse, from this client's last turn on that model. Use it when a prompt grows, before raising num_ctx, or before switching model. For what is installed use list_local_models; for what actually happened use read_my_last_turn. model: a pulled model name, "gemma4" or "gemma4:latest"; unknown names get the list of pulled ones. prompt_tokens: 1 to 2000000, your estimate of the prompt size. client: your X-LLMxRay-Client name, so the cache estimate uses your own last turn. Needs Ollama reachable; the cache line needs LLMxRay running (npx llmxray). Under a second. No model is loaded or called.

NameRequiredDescriptionDefault
modelYesA pulled model name, with or without the tag.
clientNoSubstring of the client label, for the cache estimate. Omit for the last caller.
prompt_tokensYesSize of the prompt you intend to send, in tokens, 1 to 2000000.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the read-only/idempotent annotations, it discloses dependency requirements (Ollama must be reachable; the cache line needs LLMxRay running via npx llmxray), latency (under a second), and the critical side-effect guarantee that no model is loaded or called. This is exactly the extra context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core question and followed by usage, per-parameter notes, and operational constraints. The prose is dense and slightly run-on ('the headroom left for the answer'), but every clause carries information and nothing is padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden itself, and it does so by naming the four things returned. Dependencies, latency, and non-side-effects are all covered, leaving nothing an agent needs to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real semantics: unknown model names return the list of pulled ones, prompt_tokens is 'your estimate of the prompt size', and client scopes the cache estimate to your own last turn. These clarify intent beyond the schema's terse field docs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (estimate) and resource (my prompt) and enumerates the exact outputs: context fit, headroom for the answer, whether the model sits in VRAM, and cache reuse from this client's last turn. An agent can distinguish it from read_my_last_turn or list_local_models without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit triggers ('when a prompt grows, before raising num_ctx, or before switching model') and explicitly routes to alternatives: list_local_models for what is installed, read_my_last_turn for what actually happened. Both the when and the alternatives are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_local_modelsList local modelsA
Read-onlyIdempotent

Which models Ollama has pulled on this machine, with size, parameters and quantisation. Which are loaded now and with what context. How much of each sits in video memory, and whether each pulled model would fit. Use it before choosing a model or a context length. It knows nothing about past calls. For those use diagnose_my_turns or read_my_last_turn. Returns text. A machine line (GPU, video memory, RAM) when LLMxRay is running, then "N models pulled:" and one line per model, sorted by name, ending in LOADED now, fits in VRAM, or larger than VRAM. Takes no arguments, no model name and no filter. Needs Ollama reachable. Under a second, two listing calls. No model is loaded. The answer is a snapshot: a loaded model unloads after its keep_alive, so call it right before deciding.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safe-read profile (readOnly, idempotent, non-destructive, closed-world), and the description adds substantial operational context beyond them: requires Ollama reachable, runs under a second in two listing calls, loads no model, and the result is a snapshot that goes stale after keep_alive. It even describes the output line format and sort order.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with what the tool reports, then usage, then return format and constraints. Dense and largely waste-free, though the return-format paragraph is verbose for what it conveys.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the full burden of explaining returns and does so explicitly (machine line, 'N models pulled:', per-model lines with LOADED/VRAM verdicts). Nothing needed to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so the schema baseline applies. The description confirms this by stating it takes no arguments, no model name and no filter, which prevents an agent from inventing filters, but it adds nothing beyond that confirmation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (lists Ollama models pulled on this machine) plus what is reported: size, parameters, quantisation, load state, context, VRAM fit. It explicitly distinguishes itself from siblings by noting it knows nothing about past calls, routing those to diagnose_my_turns and read_my_last_turn.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit timing guidance ("Use it before choosing a model or a context length", "call it right before deciding") and names the alternative tools for a different need (past calls). The when-not is clear: not for historical call data, and no filtering is possible.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_my_turnsList my turnsA
Read-onlyIdempotent

What happened, turn by turn: the turns your client sent through LLMxRay on port 11435, newest last, one line each: turn number, time, client, model, prompt tokens with cached share, prefill ms, generation speed, stop reason, and flags (reloaded, rejected or cut by the context, error). Four tools read your turns, one question each: diagnose_my_turns, is anything wrong (a verdict over several turns); read_my_last_turn, what exactly happened on the latest turn (one turn in full); list_my_turns, what happened turn by turn (the sequence, one line each); compare_my_turns, did a change help (before against after). Use it to see the sequence behind a finding, or to find when a change took effect before calling compare_my_turns with that many minutes. client: your X-LLMxRay-Client name; without it on a shared machine you read every tool's turns. since_minutes: 1 to 10080. limit: 1 to 100, default 20, the newest kept. The three combine. Needs LLMxRay running (npx llmxray). Under a second. No model is called.

NameRequiredDescriptionDefault
limitNoHow many of the newest turns to show, 1 to 100. Default 20.
clientNoSubstring of the client label, case-insensitive. Omit for all clients.
since_minutesNoOnly turns from the last N minutes, 1 to 10080.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the bar is lower, yet the description still adds real behavioral context: it requires LLMxRay running (npx llmxray), responds under a second, calls no model, and warns that omitting client on a shared machine exposes every tool's turns. It stops short of explaining ordering/pagination edge cases beyond 'newest last'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with what it returns, then the sibling-routing matrix, then usage, then parameter notes and operational requirements. The four-tool comparison is slightly long but directly serves tool selection, so it earns its place; every other sentence is compact.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by describing the returned line format field by field. Prerequisites (LLMxRay must be running), latency, filter interaction, and client-scoping behavior are all covered, which is everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds semantics beyond the schema: the client param is tied to the caller's X-LLMxRay-Client identity and its omission has a scoping consequence, and it states that the three filters combine. Ranges and default 20 are repeated from the schema rather than added.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb+resource (list the turns sent through LLMxRay) and enumerates exactly what each line contains: turn number, time, client, model, tokens with cached share, prefill ms, generation speed, stop reason, flags. It also positions itself against the sibling tools by naming the four turn-reading tools and the distinct question each answers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'four tools read your turns, one question each' paragraph gives explicit when-to-use routing against diagnose_my_turns, read_my_last_turn, and compare_my_turns. It goes further by naming the concrete trigger ('to see the sequence behind a finding') and the hand-off ('find when a change took effect before calling compare_my_turns with that many minutes').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_llmxray_statusRead LLMxRay statusA
Read-onlyIdempotent

Where LLMxRay runs and what it holds: the observe port and whether it listens, the Ollama it forwards to and its version, how many turns are kept of the 500 it holds, whether request text is kept on disk, and when it started. Use it first when diagnose_my_turns or read_my_last_turn answers that nothing was seen, or to confirm which Ollama your calls reach. Those tools read the turns; this one reads the server itself. Takes no arguments. Nothing can be changed through MCP; settings live on the Live page. Needs LLMxRay running (npx llmxray); if it is not reachable the answer says so and names the URL tried. Under a second.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/idempotent annotations, the description discloses that nothing can be mutated through MCP, that settings live on the Live page, that LLMxRay must be running (npx llmxray), that failures name the URL tried, and that it responds in under a second. This is genuine operational context an agent cannot infer from the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the return payload, then usage routing, then constraints and latency. Every sentence is load-bearing: the field list defines scope, the sibling comparison routes selection, and the failure/latency notes set expectations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden and does so by enumerating the status fields. Requirements, prerequisites, failure behavior, and latency are all covered, leaving nothing an agent needs to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero arguments, so the baseline is 4. The description reinforces this with 'Takes no arguments', which is a useful confirmation but adds little semantic depth beyond an empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb (reads) and enumerates the exact resources returned: observe port and listen state, Ollama target and version, turn retention, disk retention of request text, start time. The line 'Those tools read the turns; this one reads the server itself' cleanly differentiates it from diagnose_my_turns and read_my_last_turn.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when to reach for it ('Use it first when diagnose_my_turns or read_my_last_turn answers that nothing was seen') and the alternative goal ('to confirm which Ollama your calls reach'), naming the competing siblings and the diagnostic condition that selects this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_my_last_turnRead my last turnA
Read-onlyIdempotent

What exactly happened on your latest turn through LLMxRay, in numbers: prompt tokens and cached share, prefill time and speed, generated tokens and speed, stop reason, prompt make-up (system, tools, history, new message), context requested, VRAM fit, and where the prompt stopped matching the previous turn. Four tools read your turns, one question each: diagnose_my_turns, is anything wrong (a verdict over several turns); read_my_last_turn, what exactly happened on the latest turn (one turn in full); list_my_turns, what happened turn by turn (the sequence, one line each); compare_my_turns, did a change help (before against after). Use it to budget the next call. For what is installed and loaded use list_local_models. Returns text, one fact per line. Explains how to route through 11435 when no turn was seen. client: a substring of the label LLMxRay shows, which is the name you set in X-LLMxRay-Client. On a shared machine the latest turn may be another tool's, so pass your own name. Omit it when you are the only caller. Needs LLMxRay running (npx llmxray) and says so if not. Under a second, from records already kept. No model is called.

NameRequiredDescriptionDefault
clientNoPart of the client label or id, case-insensitive substring. Omit for the latest turn from any client.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare the safety profile (readOnly, idempotent, non-destructive, closed-world); the description adds traits beyond that: the LLMxRay runtime dependency ('Needs LLMxRay running (npx llmxray) and says so if not'), latency ('Under a second, from records already kept'), absence of side effects ('No model is called'), the output shape ('text, one fact per line'), and the empty-state behavior ('Explains how to route through 11435 when no turn was seen'). This is unusually complete behavioral disclosure for a read tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The fact list, sibling routing table, usage line, and parameter guidance are all load-bearing and the scope statement is front-loaded. It is dense and slightly long, and the line about routing through 11435 is terse enough to be mildly cryptic, but nothing is redundant filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by describing the return format and empty-state behavior. It covers the runtime prerequisite, expected latency, side-effect profile, and the one optional parameter's intent, leaving nothing an agent needs in order to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema already explains the substring/omit semantics, so the baseline is 3. The description goes further by explaining the rationale the schema omits: the label is the name set via X-LLMxRay-Client, and on a shared machine the latest turn may belong to another tool, so you should pass your own name. That is genuine added meaning, though it does not cover case-sensitivity or matching edge cases.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb and resource ('what exactly happened on your latest turn') and enumerates the concrete facts returned (prompt tokens and cached share, prefill time/speed, generated tokens/speed, stop reason, prompt make-up, context requested, VRAM fit, first divergence). It explicitly positions itself against all three siblings by naming each and the single question it answers, so an agent can distinguish it without opening another schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit when-to-use ('Use it to budget the next call'), an explicit when-to-use-something-else ('For what is installed and loaded use list_local_models'), and a routing table that assigns one question to each of the four turn-reading tools. The condition that selects this tool (one turn in full) versus diagnose/list/compare is stated outright.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.8.9
    • Changedcompare_my_turns1 field changed
      • removedInput schema / additionalProperties
        Removed value: -false
    • Changeddiagnose_my_turns2 fields changed
      • removedInput schema / additionalProperties
        Removed value: -false
      • addedInput schema / properties / since_minutes / maximum
        Added value: +9007199254740991
    • Changedestimate_my_prompt1 field changed
      • removedInput schema / additionalProperties
        Removed value: -false
    • Changedlist_my_turns1 field changed
      • removedInput schema / additionalProperties
        Removed value: -false
    • Changedread_my_last_turn1 field changed
      • removedInput schema / additionalProperties
        Removed value: -false
  2. 4 tool updatesv0.8.6
    • Addedcompare_my_turns
    • Addedestimate_my_prompt
    • Addedlist_my_turns
    • Addedread_llmxray_status
  3. 4 tool updatesv0.8.5
    • Removedlast_turn_metrics
    • Addedlist_local_models
    • Removedlocal_models
    • Addedread_my_last_turn
  4. 2 tool updatesv0.8.2
    • Changeddiagnose_my_turns2 fields changed
      • changedInput schema / properties / client / description
        Previous value: -"Part of the client label or id as LLMxRay shows it (for example \"VS Code Copilot\", \"python-requests\"). Omit for all clients."New value: +"Part of the client label or id as LLMxRay shows it (for example \"VS Code Copilot\", \"python-requests\", or the X-LLMxRay-Client header you sent). Case-insensitive substring. Omit for all clients."
      • changedInput schema / properties / since_minutes / description
        Previous value: -"Only turns from the last N minutes."New value: +"Only turns from the last N minutes. Omit for every turn LLMxRay has kept."
    • Changedlast_turn_metrics1 field changed
      • changedInput schema / properties / client / description
        Previous value: -"Part of the client label or id. Omit for the latest turn from any client."New value: +"Part of the client label or id, case-insensitive substring. Omit for the latest turn from any client."
  5. 3 tool updatesv0.8.0
    • First observeddiagnose_my_turns
    • First observedlast_turn_metrics
    • First observedlocal_models

TDQS

A4.6/5.0

Scored across 7 tools

Disambiguation5/5

Each of the four turn-reading tools is explicitly framed around one distinct question (verdict vs one turn full vs sequence vs before/after), and their descriptions cross-reference each other to steer selection. The prospective estimate_my_prompt is cleanly separated from the retrospective read_my_last_turn despite both touching prompt tokens. No tool pair genuinely overlaps.

Naming Consistency4/5

A predictable verb_..._noun pattern holds throughout (estimate_, list_, diagnose_, read_, compare_). Minor deviations exist: some tools carry the 'my' owner marker while list_local_models omits it, and both read_ and list_ are used for read-like operations. Still readable and largely uniform.

Tool Count5/5

Seven tools is well-scoped for a focused Ollama/LLMxRay observability server. Every tool maps to a distinct stage (pre-flight, inventory, per-turn, aggregate, server health) and none feels redundant.

Completeness4/5

The read-only observation domain is covered across pre-flight estimate, model inventory, single-turn detail, turn sequence, before/after comparison, and server status. Gaps are minor for a read-only surface (no way to change settings or inspect retained raw request text via MCP), though the description notes settings live on the Live page.

Maintenance

ActivityMaintained
ResponsivenessResponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    C
    quality
    C
    maintenance
    Professional local context management and system diagnostic tools for AI IDEs (Cursor, Trae, Antigravity, Windsurf).
    36
    84 npm
    1
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables LLM clients to inspect local dev environments—Docker container health, pnpm workspace integrity, and stuck process detection—without manual terminal copy-pasting.
    MIT
  • A
    license
    Not graded
    quality
    F
    maintenance
    Provides AI agents with real-time cognitive health monitoring, detecting context rot through token utilization, retrieval accuracy, and session fatigue analysis.
    8 npm
    13
    MIT