Opik MCP Server
This schema describes the deprecated TypeScript opik-mcp@2 server (superseded by the Python uvx opik-mcp server in the README); it lets an MCP client manage Opik prompts, projects, and traces via discrete per-entity tools.
Prompts — list, create, create new versions, get by ID, update, delete
Projects — list (with sort/workspace options), get by ID, create, update, delete
Traces — list (filter by project ID/name), get by ID, get trace statistics over a date range
Metrics — fetch metric data for a project/time window, optionally filtered by metric name
Help & docs — contextual help, API usage examples, tracing-capability info, and server config introspection
Note: the README's current server instead exposes a consolidated read / list / write / schema / read_skill toolset covering datasets, experiments, threads, scores, and Diagnostics issues — capabilities absent here.
Used for configuration through environment variables in a .env file
Hosts the repository and provides licensing information
Provides a Makefile for common operations like testing and starting the server
Used as the runtime environment for the MCP server
Implements pre-commit hooks to ensure code quality
Provides community support through the linked Slack community
Provides TypeScript language support for the MCP server implementation
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Opik MCP Serverlist my recent prompts"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Opik MCP Server
The official Model Context Protocol (MCP) server for Opik, the open-source LLM observability and evaluation platform, built by Comet. Plug your AI host (Claude Code, Cursor, VS Code Copilot, Codex, opencode, or any MCP client) directly into your Opik workspace: read traces, log scores, and save prompt versions, all from the chat.
Built for LLM engineers who already run Opik and want to drive it from the same AI assistant they code with.
You: "Which traces in project 'demo' failed today?"
Claude: → list(entity_type="trace", project_name="demo") → "Three traces failed…"
You: "Score trace 7f2e… 0.9 on helpfulness with reason 'great recovery'."
Claude: → write(score.create) → doneQuick start
One command sets Opik up in the AI clients on your machine, Claude Code
included. It finds your Opik (the one saved in ~/.opik.config, one running on
localhost, or it asks which), registers the MCP server, installs the Opik skill
pack, and checks the connection. On Opik Cloud there is no API key to handle:
your client signs in in the browser. It needs uv
and no Opik SDK:
uvx opik@latest mcp configureWithout a terminal, as from a coding agent or a script, name the client:
uvx opik@latest mcp configure --ai-client claude-code --skills--ai-client also takes codex, cursor, vscode and opencode. Run that
way it uses the Opik saved in ~/.opik.config, API key included, or one
answering on localhost, and otherwise stops and asks for --deployment.
A coding agent asked to "install opik mcp" gets the full steps, including what
to ask the user, from uvx opik-mcp@latest --help. They are the same as
Install with a coding agent below.
Registering the server by hand, with the client commands under Set it up by hand, skips the check of which Opik you use and the skill pack, so it is for clients the installer doesn't support.
Your Opik | Server | Transport | Sign-in |
Opik Cloud ( | the hosted server, run by Comet | Streamable HTTP | in the browser (OAuth); no API key |
Self-hosted Comet, open-source Opik | the local server, this package | stdio | env vars; an API key only where the deployment needs one |
Clients load MCP servers when a session starts, so start a new session afterwards.
Setup guide, troubleshooting and FAQ: comet.com/docs/opik/mcp-server.
Related MCP server: dap-mcp
Opik Cloud: the hosted server
Comet runs the server at https://www.comet.com/opik/api/v1/mcp. Your client
connects over HTTP and opens a browser sign-in the first time. There is nothing
to install, no API key, and no workspace to set: the server works in the
workspace you pick when you sign in. After adding it, start a new session and
ask: "list my Opik projects".
The Quick start command sets this up for Claude Code, Codex, Cursor, VS Code Copilot and opencode, with the skill pack. The buttons below add it to Cursor or VS Code in one click; other ways to register it by hand are under Set it up by hand.
Self-hosted and open-source Opik: the local server
opik-mcp runs on your machine: the client starts it with uvx opik-mcp and
talks to it over stdio. The Quick start command sets it up from
your ~/.opik.config or a local Opik; to register it by hand, see
Set it up by hand. Install uv once; it
fetches the package, and Python 3.13 if needed, on first use:
curl -LsSf https://astral.sh/uv/install.sh | sh # macOS / Linux
# or: brew install uvuvx opik-mcp reuses the copy uv has cached, so it starts quickly;
uv cache clean opik-mcp makes the next start fetch the newest release.
uvx opik-mcp@latest asks PyPI on every start, which adds about a second and a
half. Codex waits mcp_optional_startup_grace_ms (1 s by default) for servers
before it builds the first tool list, so a slower start can leave the tools out
of the first turn. The one-off commands in this README (opik mcp configure,
opik mcp status, --help, --check) carry @latest, so they never run a
copy uv cached months ago.
Env vars point the server at your Opik:
Your Opik | Env vars |
Open-source Opik on this machine |
|
Open-source Opik on a server |
|
Self-hosted Comet platform (the Opik UI is at |
|
Open source serves its API at
/apiand has one workspace,default, so it needs noOPIK_WORKSPACE.COMET_URL_OVERRIDEwould point the server at/opik/api, which open source does not serve.A self-hosted Comet has named workspaces. Set
OPIK_WORKSPACEto the segment after/opik/in your Opik URL; left out, reads come from your account's default workspace, with no error to say so.A self-hosted Comet that runs the MCP OAuth server (off by default) can use the hosted flow instead, at
https://<host>/opik/api/v1/mcp.Substitute every value. The server refuses a placeholder workspace such as
<your-workspace>or${input:OPIK_WORKSPACE}; a placeholder URL is sent as-is and fails to connect.OPIK_MCP_ANALYTICS_SOURCE=""in theenvblock opts a self-hosted install out of the cloud-Comet source label on telemetry events.
Before adding the server to a client, check the settings: run it with the same
env and --check. It makes one call to Opik and prints the workspace and how
many projects it sees, or what to fix with a link to the docs, and exits 0 or 1.
A client can't tell you this before a new session: Claude Code shows Connected
even with a wrong key, and Codex starts nothing until then.
OPIK_URL=http://localhost:5173/api uvx opik-mcp@latest --checkThen register it with your client: the local server, by hand.
Install with a coding agent
For an AI agent asked to install the server, as in "install opik mcp". The aim
is the server registered for the user's Opik, the Opik skill pack installed,
and the user signed in. Work out which Opik from this machine, and ask the user
only what it can't tell you: one question per message, each answer before the
next. The installer in step 3 does the whole setup in one run, so settle steps
1 and 2 before you run it. uvx opik-mcp@latest --help prints these steps.
See what is there.
uvx opik@latest mcp statusIt shows the Opik saved in
~/.opik.config(URL and workspace, never the key) and the clients that already have the server. In Claude Code, also runclaude mcp list: it shows claude.ai connectors and entries under other names. It also prints each entry's command, which for an old npx entry can hold an API key: don't repeat it.A server for
https://www.comet.com/opik/api/v1/mcpunder any name, such as a claude.ai connector, means it is registered: don't add a second one. Install the skill pack on its own withnpx skills add comet-ml/opik-skills -g --all. Go to step 4 if the server shows! Needs authentication.The installer replaces an existing
opik-mcpentry without asking, such as one for the old npx server. Ask the user before you run it.
Find the user's Opik.
Saved in
~/.opik.config: use it, and tell the user which one you are setting up. The installer reads it, API key included, so don't ask for a key.No
~/.opik.config:curl -s http://localhost:5173/api/is-alive/ping. An answer is an open-source Opik on this machine. In a sandboxed shell, such as Codex's, no answer can mean the shell has no network rather than no Opik.Neither: ask, and don't pick one yourself. An Opik URL in the project, such as
OPIK_URL_OVERRIDEin.env, makes it a yes-or-no question. Otherwise: "Which Opik do you use: Opik Cloud (www.comet.com), an open-source Opik (its URL), or a self-hosted Comet platform (its URL and workspace)?"
Run the installer for the client you run in. It installs the Opik skill pack with the server, with or without
--skills; keep the flag in a command you give the user to run later. On Opik Cloud it opens the browser sign-in and waits a minute for it, so tell the user first.uvx opik@latest mcp configure --ai-client claude-code --skills--ai-clientalso takescodex,cursor,vscodeandopencode. For an Opik saved in~/.opik.configor running on localhost, run it as it is:--deploymentsets the saved config aside, API key included. With neither, add the flag for the Opik the user named in step 2:The user's Opik
Add
Opik Cloud,
www.comet.com--deployment cloudOpen-source Opik
--deployment local, and--url <url>when it is not on localhostA self-hosted Comet platform,
/opik/in its URL--deployment self-hosted --url <url> --workspace <workspace>, withOPIK_API_KEYin the shellIf it fails, as when the Opik doesn't answer, tell the user what it printed and what to check, such as a VPN or the URL. Don't switch to another Opik.
Sign-in, for Opik Cloud. If the installer reports the server as not signed in yet, give the user
claude mcp login opik-mcpto run, or/mcpin a Claude Code session, rather than running it yourself: it waits on the browser. A claude.ai connector signs in from/mcp. The sign-in asks for a workspace; name the one saved in~/.opik.config, if any.Finish. Say what was installed and what is left. Name the skill pack: the Opik skills, installed for every project, which teach the agent to instrument, evaluate and debug with Opik. Clients load MCP servers and skills when a session starts: ask the user to start a new session, then try "list my Opik projects".
On every path:
Don't ask the user to paste an API key into the chat. The hosted server needs none; otherwise take it from the shell environment (
"$OPIK_API_KEY"), or let the user run the command.Don't guess the workspace. It is the segment after
comet.com/opik/(or/opik/on a self-hosted Comet) in the user's Opik URL.Don't print
~/.claude.jsonor~/.opik.config: both can hold API keys.uvx opik@latest mcp statusandclaude mcp listshow what you need, and the installer reads the config itself.If a permission check blocks a command, don't work around it. Give the user the command to run (in Claude Code,
!before it runs it from the prompt).
For a client the installer doesn't know, register by hand with the commands in Set it up by hand: the hosted server for Opik Cloud, the local server for the rest.
Before registering the local server, run it with the env you will pass and
--check, withOPIK_API_KEYexported in the shell rather than written on the command:OPIK_URL=http://localhost:5173/api uvx opik-mcp@latest --check. It exits 0 and names the workspace it sees, or prints what to fix. Once registered, Claude Code reports Connected even with a wrong key, and Codex has no check.Claude Code refuses to add over an existing
opik-mcpentry, so remove it first withclaude mcp remove opik-mcp --scope user; Codex'saddreplaces it.After registering, in Claude Code, read only the status line:
claude mcp get opik-mcp | grep Status. The full output prints theenvblock, API key included. The hosted server shows! Needs authenticationuntilclaude mcp login opik-mcphas run. Codex has nothing that starts the server before a session;codex mcp get opik-mcpshows what was stored, with env values masked.
Set it up by hand (no Opik check, no skill pack)
The Quick start command does all of this for Claude Code, Codex, Cursor, VS Code Copilot and opencode, and also checks which Opik you use and installs the skill pack. Register by hand only for another client, or to see what the installer writes. A coding agent installing for a user runs the installer: see Install with a coding agent.
Hosted server (Opik Cloud)
Claude Code, by hand
claude mcp add --scope user --transport http opik-mcp https://www.comet.com/opik/api/v1/mcpThen sign in from /mcp → Authenticate in a Claude Code session. It opens
a browser, so the person at the keyboard does it; from their own terminal,
claude mcp login opik-mcp does the same. --scope user makes the server available in every project; without
it, Claude Code registers it for the current directory only. Over SSH,
claude mcp login opik-mcp --no-browser prints the sign-in URL to open on your
own machine; the last step needs an interactive terminal (ssh -t).
Codex, by hand
codex mcp add opik-mcp --url https://www.comet.com/opik/api/v1/mcpadd opens the sign-in; codex mcp login opik-mcp opens it again. In
~/.codex/config.toml the same server is:
[mcp_servers.opik-mcp]
url = "https://www.comet.com/opik/api/v1/mcp"Cursor, by hand
Use the button under the hosted server, or add to
~/.cursor/mcp.json:
{
"mcpServers": {
"opik-mcp": {
"url": "https://www.comet.com/opik/api/v1/mcp"
}
}
}VS Code Copilot, by hand
Use the button under the hosted server, or add to
.vscode/mcp.json in your workspace or to your
user mcp.json (MCP: Open User Configuration):
{
"servers": {
"opik-mcp": {
"type": "http",
"url": "https://www.comet.com/opik/api/v1/mcp"
}
}
}Other clients
Any client that takes a URL can use the hosted server:
npx add-mcp https://www.comet.com/opik/api/v1/mcp --name opik-mcpA client that can only start local commands can reach it through
npx -y mcp-remote https://www.comet.com/opik/api/v1/mcp.
Opik Cloud with an API key
Where nobody can complete the browser sign-in, as when an agent runs unattended
from a script or a client has no MCP OAuth support, use an API key from
comet.com/api/my/settings/ instead,
with the local server
pointed at Opik Cloud:
claude mcp add --scope user opik-mcp \
--env OPIK_API_KEY="$OPIK_API_KEY" \
--env OPIK_WORKSPACE=<workspace> \
-- uvx opik-mcpThe other clients take the same two variables in their env block. Set
OPIK_WORKSPACE to the segment after comet.com/opik/ in your Opik URL
(https://www.comet.com/opik/acme-ai/projects → acme-ai). Left out, the
server sends default, which Comet resolves to your account's default
workspace, so reads can come from the wrong workspace without an error.
Local server (self-hosted and open-source Opik)
The examples use a local open-source Opik. For a self-hosted Comet, swap in the three variables from the table. After adding the server, start a new session and ask: "list my Opik projects".
Claude Code, by hand
claude mcp add --scope user opik-mcp --env OPIK_URL=http://localhost:5173/api -- uvx opik-mcpOn a self-hosted Comet, with the key in your shell's OPIK_API_KEY:
claude mcp add --scope user opik-mcp \
--env COMET_URL_OVERRIDE=https://<host> \
--env OPIK_WORKSPACE=<workspace> \
--env OPIK_API_KEY="$OPIK_API_KEY" \
-- uvx opik-mcpOr edit ~/.claude.json directly:
{
"mcpServers": {
"opik-mcp": {
"type": "stdio",
"command": "uvx",
"args": ["opik-mcp"],
"env": {
"OPIK_URL": "http://localhost:5173/api"
}
}
}
}claude mcp get opik-mcp shows ✔ Connected once the server starts. That does
not prove the URL or key are right, because only a tool call reaches Opik;
--check (the local server) does. It also prints the env block, API key included.
Codex, by hand
codex mcp add opik-mcp --env OPIK_URL=http://localhost:5173/api -- uvx opik-mcpOr edit ~/.codex/config.toml:
[mcp_servers.opik-mcp]
command = "uvx"
args = ["opik-mcp"]
env = { OPIK_URL = "http://localhost:5173/api" }
# On a self-hosted Comet, forward the key from the environment Codex starts in,
# so it stays out of this file:
# env_vars = ["OPIK_API_KEY"]
# Time allowed for the server to start (default 10 s); the first start
# downloads the package.
startup_timeout_sec = 30Codex gives a local server only the variables in env and the names in
env_vars, so a key exported in your shell does not reach it otherwise.
Cursor, by hand
Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project), or use
Cmd+Shift+J → Features → Model Context Protocol:
{
"mcpServers": {
"opik-mcp": {
"type": "stdio",
"command": "uvx",
"args": ["opik-mcp"],
"env": {
"OPIK_URL": "http://localhost:5173/api"
}
}
}
}Cursor 60s timeout. Cursor enforces a hard tool-call timeout that doesn't reset on progress notifications. See Known host limits.
VS Code Copilot, by hand
Add to .vscode/mcp.json in your workspace, or to your user mcp.json
(MCP: Open User Configuration):
{
"servers": {
"opik-mcp": {
"type": "stdio",
"command": "uvx",
"args": ["opik-mcp"],
"env": {
"OPIK_URL": "http://localhost:5173/api"
}
}
}
}MCP Inspector (manual testing)
OPIK_URL=http://localhost:5173/api npx @modelcontextprotocol/inspector uvx opik-mcpComing from npx opik-mcp?
The TypeScript server (npm opik-mcp@2) is deprecated and stops serving
requests on 2026-11-15. For a new install, run the
Quick start command; uvx opik-mcp is the command a client runs
to start the server, not an installer. To move an existing config on Opik Cloud,
switch to the hosted server and drop the API key. Otherwise, in your MCP client
config, replace npx -y opik-mcp with uvx opik-mcp. Some env vars were renamed and command-line flags are no
longer read: see the migration
guide.
Support policy: DEPRECATED.md.
The TypeScript source is at the git tag legacy-typescript-final.
Tools
opik-mcp exposes a small, outcome-oriented surface that covers the full
lifecycle (read → annotate → curate → author → iterate).
Tool | Purpose |
Universal read by id / name / | |
Universal list with optional name filter + pagination | |
Universal write — log traces/spans, score, comment, save prompts, manage datasets & experiments | |
Introspect write-operation schemas (used by the LLM to construct valid payloads) | |
| Read one of the Opik agent skills bundled with this server |
read
One tool for any "show me X" question. Takes an entity_type plus an id
(UUID or, for nameable types, a name) or a full opik:// URI. Composite reads
(trace, prompt, thread, agent_insights_issue) inline their children so
a single call returns the full picture.
The record you name comes back whole. Inlined children do not: their bodies
are fetched with the backend's truncate=true, so a field over ~10 KB is cut
in ClickHouse and base64 images are replaced with "[image]" — one attachment
echoed across 200 spans would otherwise cost more than everything else in the
read. The answer says so in spanBodies / messageBodies, and any child is
whole again through its own read("span", id) or read("trace", trace_id),
which hit endpoints that have no truncate parameter at all.
An inlined collection is also bounded in length: 200 spans, 200 turns, 100
prompt versions. Past that, spansTruncated / messagesTruncated /
versionsTruncated is true and a moreSpans / moreMessages /
moreVersions line beside it carries the count and the exact list(...) call
that continues from where the inlined part stopped.
Supported entities: project, trace, span, dataset, dataset_item,
experiment, prompt, thread, agent_insights_issue. Name-based lookup is
available for project, experiment, prompt, dataset (slower — two API
calls — and may return multiple matches). thread and agent_insights_issue
are project-scoped: pass project_id or project_name, or a link/URI that
carries the project. dataset and dataset_item were called test_suite and
test_suite_item before; the old names still resolve, but they are not
advertised and new code should use the new ones.
read(entity_type="trace", id="7f2e3c8a-…")
read(entity_type="project", id="demo") # name lookup
read(entity_type="trace", id="opik://traces/7f2e3c8a-…")
read(entity_type="agent_insights_issue", id="<issue-uuid>", project_id="<project-uuid>")
read(
entity_type="agent_insights_issue",
id="https://www.comet.com/opik/<ws>/projects/<pid>/diagnostics?issue=<id>",
)A link copied from the Opik UI works as the id: a thread link or a
Diagnostics page link carries the project, so no project_id is needed and
the entity type is taken from the link.
A project read answers "how is my project doing" in one call. It returns
{project, summary, vocabulary, contains, url}: the record, then the four
figures the Logs page shows as cards (trace count, error rate, average
duration, total cost) for the last 7 days against the 7 before, SDK traffic
only, as on screen. since / until move that window; since="30d" is what
the UI opens on. A rate or an average over a period with no traces comes back
as null, because 0% errors on a week with no traffic reads as a healthy week.
vocabulary is the map you need before you can ask anything else: the
project's feedback score names, its token usage keys, and the automation rules
scoring its traces. These are the names that go into a filter or into
series= below, and guessing them returns an empty page that reads like good
news. Score names and rules are capped, always report the true total, and name
the call that returns the rest; usage keys are listed in full, since nothing
else enumerates them. contains names the freshest experiment, dataset, prompt
version and optimization run, so "what has been happening here" does not need
four more calls. A part that failed to load says so instead of looking empty,
and an empty one is omitted.
An agent_insights_issue read returns {issue, example_trace_ids, details}:
the Diagnostics issue record (name, description, cause, suggested fix,
severity, status), the deduplicated ids of the traces that exhibit it (the
same sample the Diagnostics page shows — open one with read("trace", id)),
and the per-day breakdown. Trace bodies are not inlined, so the read stays one
backend call. since / until narrow the per-day rows; the default is
all-time. When the server knows the Opik URL and the session's workspace, the
read also carries url (the issue's Diagnostics page) and trace_url_template
(a deep link for any of the example traces), so the assistant can hand you
something clickable; under an OAuth session whose workspace could not be
resolved the links are omitted rather than guessed.
Traces themselves carry no URL — a link for one is not derivable from the
fields a read or list returns, and a guessed shape 404s. The session
instructions name a template for it instead,
.../v1/session/redirect/projects/?trace_id={trace_id}&path=..., so the
assistant fills in an id and hands you a link. It goes through opik-backend's
redirect, which resolves the project and the workspace from the trace, so it
works where a direct project URL cannot, an OAuth session with an unresolved
workspace included. It is the same link the Python SDK prints for a trace.
list
Browse or search a collection with pagination. Project-scoped types (trace,
span, thread, agent_insights_issue, dataset_item, prompt_version)
need their parent: a project UUID or name, a dataset UUID, or a prompt UUID.
list(entity_type="experiment", page=1, size=25)
list(entity_type="experiment", name="rerank") # name substring filter
list(entity_type="agent_insights_issue", project_name="demo") # open Diagnostics issues
list(entity_type="agent_insights_issue", project_id="<uuid>", status="resolved")
list(entity_type="trace", project_name="demo") # latest traces of one project
list(
entity_type="trace", project_name="demo", filters="error_info is_not_empty AND duration > 5000"
)
list(
entity_type="span",
project_name="demo", # spans across the whole project
filters='type = "llm" AND usage.total_tokens > 10000',
)
list(
entity_type="thread",
project_name="demo",
filters="number_of_messages > 20 AND feedback_scores.helpfulness < 0.5",
)
list(entity_type="experiment", filters='dataset_id = "<dataset-uuid>" AND tags contains "baseline"')Filters. trace, span, thread, experiment and dataset_item take an
OQL string, the same grammar as the SDK's search_traces(filter_string=…):
<field>[.<key>] <op> <value> [AND ...]
ops: = != > >= < <= contains not_contains starts_with ends_with is_empty is_not_empty in not_inStrings go in double quotes, numbers are bare, duration is in milliseconds,
dates are ISO-8601 instants with a timezone ("2026-09-08T10:00:00Z").
Scores and dictionaries take a key: feedback_scores.accuracy < 0.5,
metadata.environment = "prod". AND is the only connector.
Like the UI's Logs page, trace, span and thread lists add source = "sdk" so
evaluator, playground and experiment traces stay out of the way; name source
yourself to see them. The first output line echoes the filter that was applied.
A bad filter fails before reaching the backend with what is needed to fix it:
the position of a syntax error, the closest field name, the valid operators for
the field's type, or the expected value format. Fields with a closed set of
values (source, span type, thread status, visibility_mode) are checked
against it too, every element of an in list included. source is the one the
backend validates itself, and it answers an unknown value with a 500 rather
than a 400, so source = "SDK" would otherwise be an opaque server error for a
capital letter. The rest are compared as strings and answer with an empty page,
which reads as "no matches" when it means "no such value". Ask
schema("list.trace") (or list.span, list.thread, list.experiment) for
the full field reference, accepted values included.
Finding one case in a dataset. list(entity_type="dataset_item", dataset_id=…) filters on the case itself: data.<key> for the keys the
dataset was built with, full_data for a substring of the whole payload (a
full scan — name a key when you can), plus id, tags, source, trace_id,
span_id and the timestamps. data.<key> takes the six string operators only
(=, !=, contains, not_contains, starts_with, ends_with); the
backend answers a comparison with a 400, so this one is refused before the
call. The endpoint has no sorting and no free-text search — sort is refused
rather than dropped. read(entity_type="dataset_item", id=…) returns one case
whole, which is how a value the table cut is read back.
list(entity_type="dataset_item", dataset_id="<uuid>", filters='data.question contains "install"')
list(
entity_type="dataset_item", dataset_id="<uuid>", filters='trace_id = "<trace-uuid>"'
) # the case made from that trace
read(entity_type="dataset_item", id="<item-uuid>") # the case, uncutWith experiment_ids the same list is the comparison instead — the cases with
each run attached — and it filters on the runs (feedback_scores.<name>,
output, duration). The two are different field sets on two backend
endpoints: schema("list.dataset_item_case") is the dataset's own cases,
schema("list.dataset_item") the comparison.
Sort. trace, span, thread and experiment take
sort="<field> [asc|desc]", desc by default and one field only:
sort="duration desc", sort="total_estimated_cost",
sort="feedback_scores.accuracy asc", sort="usage.total_tokens". The field is
checked against the entity's sortable list before the call, because the backend
silently ignores fields it cannot sort by. On very large workspaces the backend
drops sorting altogether; the header says so when that happens.
dataset_item sorts only as a comparison (with experiment_ids): the items
endpoint takes no sorting parameter, so a sort on a plain listing is refused
rather than dropped.
Time window and search. trace, span and thread take since and
until, each a relative span ("30m", "1h", "7d") or an ISO-8601 instant
with a timezone, so "the last hour" needs no clock arithmetic. The window is by
record creation time, which is cheap for the backend and agrees with
start_time within seconds for live traffic. For an exact bound, put
start_time in filters. The same three types take search, free text matched
anywhere in id, name, input, output, metadata, tags and thread id. Search scans
the whole project on the backend, so the first call on a large project can take
tens of seconds. Those calls get a 60-second timeout. Adding since makes them
fast again.
Reading the table. Durations are labelled duration_ms / ttft_ms and
shown as whole milliseconds; the field stays duration in filters and
sort. Timestamps are shown to the second and costs as plain decimals.
Project rows carry last_updated_trace_at so you can see which project has
live traffic; thread rows carry the first message. An empty page under a time
window says when the project's last trace landed, and an empty page under the
default source = "sdk" says how to see the other sources. A misspelled
project_name comes back with the closest existing name.
list(
entity_type="trace",
project_name="demo",
since="1h",
filters="error_info is_not_empty",
sort="duration desc",
)
list(entity_type="trace", project_name="demo", search="order-42")Diagnostics issues. agent_insights_issue is the Diagnostics page over
the MCP: the recurring failures Opik's Diagnostics job grouped for a project,
ranked as the UI ranks them (most recently seen first). Columns are severity,
status, total_occurrences (all-time sum), latest_count (the most recent
report day, the number the issue's own description refers to) and last_seen.
Open issues are listed by default; pass status="resolved" or "closed" for
the rest. read and list also answer to issue, which is what the UI calls
these; the long name is the one in the entity_type enum, so that one entity
does not appear there twice. Counts are all-time so they match the UI; the same since / until
as for traces narrow the window, truncated to UTC report days because
Diagnostics aggregates per day.
An empty list says why it is empty, because "nothing is broken" and "nobody turned Diagnostics on" read the same otherwise. There are five states: Diagnostics is unavailable on this deployment, not enabled for this project, turned off, enabled but not scanned recently, or enabled and clean with the time of the last scan. The ones you can act on name the call to make, and every state links the project's Diagnostics page.
A non-empty list dates itself. The issues are whatever the last scan grouped,
so the reply ends with Report covers data through <time>, and when the window
you asked about runs past that, it names the uncovered tail and how to close
it: a trigger when a rescan reaches back far enough, otherwise raw traces with
the since it gives you. Ask for a week on a project scanned nightly and the
last day is missing from the grouped answer; this is what says so.
write("agent_insights_job.enable", {"project_name": "demo"}) turns Diagnostics
on. It scans daily from then on, and calling it again is safe.
write("agent_insights_job.trigger", …) scans the last 24 hours now, without
waiting for the nightly run. Both take the permission that reading issues takes,
and both refuse where the deployment has no Diagnostics.
An issue moves through its lifecycle with
write("agent_insights_issue.resolve", {"issue_id": "<uuid>", "project_name": "demo"})
— dealt with — or …close for one not worth acting on, and …reopen to put
either back on the open list. All three take the same permission and answer
with a link to the view the issue moved to, since a resolved issue is no longer
on the default page. Whether a failure is fixed is a judgment call, so these
are for when you ask: the assistant has no business tidying the list while
triaging it.
Metrics over time. project_metric charts one metric for a project as a
table of time buckets: trace, span and thread counts, durations, error rates,
costs, token usage and feedback scores. It answers the question that follows
the overview, which is when something changed.
list(entity_type="project_metric", project_name="demo", metric_type="trace_count")
list(
entity_type="project_metric",
project_name="demo",
metric_type="trace_error_rate",
since="14d",
interval="daily",
)
list(
entity_type="project_metric", project_name="demo", metric_type="span_count", breakdown="model"
) # one column per model
list(
entity_type="project_metric",
project_name="demo",
metric_type="span_duration",
breakdown="model",
series="p99",
) # the p99 of each modelRows are time buckets, not records, so page, size and sort are refused
rather than ignored. interval is hourly, daily, weekly or total;
left out, it follows the window the way the Metrics tab does — hourly up to 3
days, daily up to 30, weekly beyond — so a default chart is a few dozen rows
whatever the range, and an hourly month (721 rows) is something you ask for.
since / until take the same forms as everywhere else and default to the
last 7 days. filters uses the fields of whichever entity the metric is
about, so a span metric is filtered by span fields.
breakdown splits each bucket by tags, name, error_info, error_type,
model, provider, span_type, guardrail_name or metadata.<key>. Not
every metric accepts every one of those, and seven accept none at all; the tool
knows which and says so before calling the backend, naming a metric that does
answer the same question where one exists. Three families come back as several
series at once (a duration as p50/p90/p99, a feedback score per name, token
usage per key), and the backend charts one of them at a time when grouping, so
series= picks it: a percentile, a score name, or a usage key. Duration
defaults to p50 and token usage to total_tokens, and whichever was used is
echoed on the first line.
Empty buckets are left out and counted underneath, so a quiet month is a few rows instead of a column of zeros, and a rate over a bucket with no traces is absent rather than reported as zero.
Ask schema("list.project_metric") for the metric table, the intervals and the
per-metric grouping matrix.
A project's names. score_name lists the feedback score names recorded in
a project and online_rule the automation rule evaluators configured on it,
which is where most of those names come from. Both are the same lists
read("project", …) carries, in full and paginated, for when the capped
version in the overview is not enough.
list(entity_type="score_name", project_name="demo")
list(entity_type="online_rule", project_name="demo")write
Universal write dispatcher. Pass operation + data and the dispatcher
validates the payload, applies the right REST verb, and returns the
backend response.
Operations:
Operation | What it does |
| Log a single trace (or a batch). Parent for spans / scores / comments. |
| Finalize or amend an existing trace. |
| Log a span on an existing trace (or a batch). |
| Attach a numeric feedback score to a trace, span, or thread. |
| Attach a free-text comment to a trace, span, or thread. |
| Save a new prompt version (creates the prompt by name if missing). |
| Create a dataset — |
| Upsert items into a dataset (always the envelope shape). |
| Create an experiment scoped to a dataset. |
| Attach trace + dataset_item rows to an experiment. |
| Close a thread (mark it inactive). Pass |
| Reopen a closed thread. Pass |
| Turn Diagnostics on for a project (daily scans, safe to repeat). |
| Run a Diagnostics scan now, over the last 24 hours. |
| Mark a Diagnostics issue dealt with (ask the user first). |
| Mark a Diagnostics issue not worth acting on (ask the user first). |
| Put a resolved or closed Diagnostics issue back on the open list. |
write(
operation="score.create",
data={
"target": "trace",
"target_id": "7f2e3c8a-…",
"name": "helpfulness",
"value": 0.9,
"reason": "great recovery",
},
)schema
Inspect the exact JSON shape and required fields of any write operation before
you call it — useful when you're not sure what data should look like. Returns
the schema, OAuth scope, and one validated example. Pure lookup, no backend
call.
schema(operation="score.create")
schema(operation="prompt_version.save")The same tool answers list.trace, list.span, list.thread and
list.experiment with the list tool's reference for that entity: every
filterable field with its type and valid operators, the sortable fields, whether
a time window and free-text search apply, and two example filters.
schema(operation="list.trace")Configuration
These configure the local server; every setting is an environment variable. The hosted server on Opik Cloud takes none of them.
Identity / endpoint
Variable | Default | Notes |
| — | API key, for a self-hosted Comet, or for Opik Cloud without the hosted server. Open-source Opik needs none unless the deployment adds authentication. |
| unset | Workspace name. On cloud with an API key, unset sends |
| — | Deprecated alias for |
| unset | Optional workspace UUID. Stamped into analytics events when set, and takes precedence over the resolved one. Rarely needed — OAuth installs get the UUID from the token automatically. |
|
| Set to your self-hosted Comet host, or |
| derived from | Set it for open-source Opik, which serves its API at |
| unset | When set, the per-session |
Server / transport
Variable | Default | Notes |
|
|
|
|
| uvicorn bind host ( |
|
| uvicorn bind port ( |
|
|
|
| unset | OAuth Authorization Server URL, advertised in |
| unset | Canonical public URI of this server, advertised as |
|
| How long a "valid" answer from opik-backend's token introspection is trusted before the next request on the same OAuth token asks again. Bounds the backend load added by per-request validation and the window in which an expired token is still forwarded (that window also ends on the first 401 the backend returns). Capped by the token's own |
|
| stderr logger threshold. |
Choosing a transport
Two bearer shapes, two contracts on HTTP transport. An opik_mcp_at_… OAuth
access token is validated on every request against opik-backend's token
introspection endpoint (cached, see OPIK_MCP_OAUTH_VALIDATION_CACHE_TTL_S);
an expired or revoked token gets an HTTP 401 with
WWW-Authenticate: Bearer error="invalid_token", which is what MCP hosts key
their silent refresh_token grant on. An Opik API key is not validated
locally: it is forwarded verbatim to opik-backend, which is its single point
of enforcement. Pick the transport by deployment shape:
Scenario | Transport |
Opik Cloud | Nothing to run: the hosted server is this server over HTTP, run by Comet |
MCP client and Opik on the same machine (local OSS install) | stdio (recommended — simplest, no port, no OAuth setup) |
Local MCP client → self-hosted Opik | stdio with the env vars for the deployment, or HTTP with OAuth ( |
opik-mcp served behind the same edge as opik-backend | HTTP — bearers are validated by the backend per request |
Note for local OSS installs: the OSS backend does not authenticate requests,
so an HTTP opik-mcp in front of it is as open as the OSS REST API itself.
Keep the default 127.0.0.1 bind (and prefer stdio) on shared networks.
Known host limits
Hosts differ in how long they let a single tool call run:
Claude Code — no documented tool-call timeout. Recommended.
Cursor — hard 60s timeout that does not reset on progress (upstream bug).
MCP Inspector —
MAX_TOTAL_TIMEOUTbounds total duration (default 60s). Raise it in the Inspector UI for long operations.
If a call gets stuck, set OPIK_MCP_LOG_LEVEL=DEBUG for the full request log.
Troubleshooting
Does the setup work? Run the server with the env from its client config and
--check: OPIK_URL=http://localhost:5173/api uvx opik-mcp@latest --check. It
names the workspace and the projects it sees, or says what to fix. The server's
own errors also say what to change, and link the docs page for it.
OPIK_API_KEY isn't picked up — the var isn't reaching the server
process. In Claude Code / Cursor / VS Code, env vars only apply when inside
the env block of the MCP server config, not your shell; Codex also forwards
the names listed in env_vars. Start a new session after editing, since
clients read the config when a session starts.
Requests go to /opik/api on an open-source Opik — COMET_URL_OVERRIDE is
for a self-hosted Comet platform. Open source serves its API at /api: set
OPIK_URL=http://localhost:5173/api (or https://<host>/api) instead. The
server warns about this at startup when the host is this machine, and --check
answers "No Opik API at …".
Cursor call times out at 60s — Cursor's known bug, not opik-mcp. Either
narrow the call (smaller size, a tighter window), or run the same operation
on Claude Code which has no hard cap.
Server not showing, sign-in not opening, wrong workspace, uvx not found.
These are covered in the troubleshooting section of the docs.
uvx opik@latest mcp status lists every client that has the server configured
and whether its config has drifted.
Development
git clone git@github.com:comet-ml/opik-mcp.git
cd opik-mcp
make install # uv sync --locked --extra dev
make check # lint + typecheck + test
make run-dev # uvicorn with --reload + DEBUG logs
make inspect # MCP Inspector against the running serverCommon targets:
Target | What it does |
|
|
| Run the MCP server (stdio by default). |
| Run with DEBUG logging + uvicorn |
| Run via |
| Launch MCP Inspector against a running server. |
|
|
|
|
|
|
|
|
|
|
Repo layout:
opik-mcp/
├── src/opik_mcp/ ← server, tools, analytics
├── tests/ ← pytest suites
├── scripts/ ← live-BE smoke + MCP-session smoke
├── legacy/typescript/ ← migration guide for the deprecated v2 TS server (source: tag `legacy-typescript-final`)
├── pyproject.toml
└── MakefileGet help
Open an issue for bugs and feature requests
Opik docs for SDK / backend documentation
Comet community Slack for questions
License
Apache-2.0.
Available Tools
19 toolscreate-projectCInspect
Create a new project/workspace
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | Description of the project | |
| name | Yes | Name of the project | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Create') but doesn't cover critical aspects like required permissions, whether the creation is idempotent, what happens on conflicts, or the expected response format. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words—it directly states the tool's purpose without unnecessary elaboration. It's appropriately sized and front-loaded, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a mutation tool ('Create') with no annotations and no output schema, the description is insufficient. It doesn't explain what a 'project' or 'workspace' entails in this system, how to handle errors, or what the tool returns upon success, leaving the agent with incomplete context for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no parameter-specific information beyond what's already in the input schema, which has 100% coverage with clear descriptions for all three parameters. This meets the baseline of 3, as the schema adequately documents the parameters, but the description doesn't provide additional context like examples or usage notes.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new project/workspace' clearly states the verb ('Create') and resource ('project/workspace'), making the purpose immediately understandable. However, it doesn't distinguish this tool from its sibling 'create-prompt' or explain what differentiates a 'project' from a 'workspace' in this context, which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'update-project' or 'list-projects', nor does it mention prerequisites or constraints. While the verb 'Create' implies it's for new entities, there's no explicit comparison to siblings or context for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create-promptCInspect
Create a new prompt
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Name of the prompt |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. 'Create a new prompt' implies a write/mutation operation but provides no details about permissions needed, whether creation is idempotent, what happens on conflicts, or what the response contains. This leaves significant behavioral gaps for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just three words. While it's under-specified in content, it's not verbose or poorly structured. Every word earns its place in this minimal description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with no annotations and no output schema, the description is inadequate. It doesn't explain what a 'prompt' is in this context, what fields beyond 'name' might be set by default, what the creation response looks like, or how this differs from similar tools. The description fails to provide necessary context for effective tool use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents the single 'name' parameter. The description adds no additional parameter context beyond what's in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no parameter information in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new prompt' is a tautology that restates the tool name without adding specificity. It doesn't distinguish this tool from sibling tools like 'create-prompt-version' or explain what kind of prompt is being created. The purpose is minimally stated but lacks meaningful differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives like 'create-prompt-version' or 'update-prompt'. There's no mention of prerequisites, constraints, or appropriate contexts for invoking this creation tool versus other prompt-related operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create-prompt-versionCInspect
Create a new version of a prompt
| Name | Required | Description | Default |
|---|---|---|---|
| commit_message | Yes | Commit message for the prompt version | |
| name | Yes | Name of the original prompt | |
| template | Yes | Template content for the prompt version |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool creates a new version but doesn't explain what that entails—whether it's a write operation, requires specific permissions, affects existing prompts, or has side effects like triggering notifications. For a mutation tool with zero annotation coverage, this lack of detail is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action ('Create a new version'), making it easy to parse quickly. Every word earns its place, and there's no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a versioning operation (a mutation with potential side effects), no annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like permissions, idempotency, or error handling, nor does it hint at return values. For a tool that modifies data, this leaves critical gaps for an AI agent to understand its full context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents the three parameters (name, template, commit_message). The description adds no additional meaning beyond what's in the schema, such as explaining how 'name' relates to the original prompt or what 'commit_message' is used for. With high schema coverage, a baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't need to.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('new version of a prompt'), making the purpose immediately understandable. It distinguishes from siblings like 'create-prompt' (which creates a new prompt rather than a version) and 'update-prompt' (which might modify an existing prompt without versioning). However, it doesn't specify what constitutes a 'version' (e.g., whether it's a snapshot, revision, or branch), leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing prompt), compare to siblings like 'update-prompt' or 'create-prompt', or indicate scenarios where versioning is appropriate (e.g., for tracking changes, experimentation, or deployment). Without this, users must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete-projectCInspect
Delete a project
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | ID of the project to delete | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but discloses nothing beyond the basic action. It fails to address critical behavioral traits: whether deletion is permanent or reversible, required permissions, side effects (e.g., on associated data), error conditions, or response format. For a destructive operation, this lack of transparency is severe.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence, 'Delete a project', which is front-loaded and wastes no words. While under-specified, it efficiently communicates the core action without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (destructive operation with 2 parameters), lack of annotations, and no output schema, the description is incomplete. It does not compensate for missing behavioral context, usage guidelines, or output expectations, leaving significant gaps for an AI agent to understand and invoke the tool safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear parameter documentation in the schema itself. The description adds no parameter semantics beyond what the schema provides, such as explaining 'projectId' format or 'workspaceName' usage. However, the baseline score of 3 is appropriate since the schema adequately covers parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a project' is a tautology that restates the tool name without adding meaningful context. It specifies the verb 'Delete' and resource 'project', but lacks distinction from sibling tools like 'delete-prompt' or details about what deletion entails. This minimal statement fails to clarify scope or differentiate from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites (e.g., project existence), exclusions (e.g., irreversible effects), or comparisons to sibling tools like 'update-project' or 'list-projects'. The description offers no context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete-promptCInspect
Delete a prompt
| Name | Required | Description | Default |
|---|---|---|---|
| promptId | Yes | ID of the prompt to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action is destructive ('Delete') but doesn't elaborate on consequences (e.g., permanent deletion, no undo), permissions required, or error conditions. This is a significant gap for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at three words, front-loading the essential action and resource without any wasted text. Every word earns its place, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's destructive nature, no annotations, and no output schema, the description is incomplete. It lacks critical information about behavioral traits (e.g., irreversibility), error handling, or what happens post-deletion. For a deletion tool, this leaves significant gaps in understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no parameter semantics beyond what the schema provides. Since schema description coverage is 100% (the 'promptId' parameter is fully documented in the schema), the baseline score is 3. The description doesn't compensate with additional context like format examples or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('a prompt'), making the tool's purpose immediately understandable. It distinguishes itself from siblings like 'delete-project' by specifying the resource type, though it doesn't explicitly contrast with other deletion tools. The description avoids tautology by not merely restating the name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a prompt ID), exclusions (e.g., cannot delete if in use), or sibling tools like 'delete-project' for different resources. Usage is implied by the action but lacks explicit context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-metricsCInspect
Get metrics data
| Name | Required | Description | Default |
|---|---|---|---|
| endDate | No | End date in ISO format (YYYY-MM-DD) | |
| metricName | No | Optional metric name to filter | |
| projectId | No | Optional project ID to filter metrics | |
| projectName | No | Optional project name to filter metrics | |
| startDate | No | Start date in ISO format (YYYY-MM-DD) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure but fails completely. 'Get metrics data' reveals nothing about whether this is a read-only operation, whether it requires authentication, what rate limits might apply, what format the data returns in, or any other behavioral characteristics. For a data retrieval tool with zero annotation coverage, this is a critical gap that leaves the agent with no understanding of how the tool behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just three words. While this represents severe under-specification in terms of content, from a pure conciseness perspective it contains zero wasted words and is front-loaded with the core action. Every word earns its place, even though that place is inadequate for proper tool understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a 5-parameter tool with no annotations and no output schema, the description 'Get metrics data' is completely inadequate. It provides no information about what metrics are available, what system they come from, what the return format looks like, or any behavioral characteristics. For a data retrieval tool of this complexity, the description fails to provide the minimal contextual information needed for an agent to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, meaning all 5 parameters are well-documented in the input schema itself. The description adds absolutely no additional parameter information beyond what's already in the schema. According to the scoring rules, when schema coverage is high (>80%), the baseline score is 3 even with no parameter information in the description, which applies here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get metrics data' is a tautology that essentially restates the tool name 'get-metrics'. It provides no specific information about what kind of metrics, from what system, or what scope. While it includes a verb ('Get') and resource ('metrics data'), it lacks any distinguishing details that would help differentiate it from potential sibling tools or clarify its specific function beyond the obvious.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides absolutely no guidance on when to use this tool versus alternatives. There are no mentions of prerequisites, appropriate contexts, or comparisons to sibling tools like 'get-trace-stats' or 'get-trace-by-id' that might handle similar data. The agent receives no help in determining when this specific metrics retrieval tool is appropriate versus other data-fetching tools in the server.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-opik-examplesCInspect
Get examples of how to use Opik Comet's API for specific tasks
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | The task to get examples for (e.g., 'create prompt', 'analyze traces', 'monitor costs') |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves examples but doesn't describe what the output looks like (e.g., format, structure), whether it's a read-only operation, or any limitations like rate limits or authentication needs. This leaves significant gaps for an AI agent to understand how to handle the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly, which is ideal for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., code snippets, documentation), how examples are structured, or any behavioral traits. For a tool with no structured data beyond the input schema, the description should provide more context to help an AI agent use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'task' parameter clearly documented. The description doesn't add any additional meaning beyond what the schema provides, such as explaining the semantics of example tasks or providing usage context. With high schema coverage, the baseline score of 3 is appropriate, as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Get') and resource ('examples of how to use Opik Comet's API'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'get-opik-help' or 'get-opik-tracing-info', which might provide related but different information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when it's appropriate (e.g., for learning API usage) or when not to use it (e.g., for direct API calls), nor does it reference sibling tools like 'get-opik-help' that might serve similar purposes.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-opik-helpBInspect
Get contextual help about Opik Comet's capabilities
| Name | Required | Description | Default |
|---|---|---|---|
| subtopic | No | Optional subtopic for more specific help | |
| topic | Yes | The topic to get help about (prompts, projects, traces, metrics, or general) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves help information, implying a read-only operation, but doesn't mention any behavioral traits like rate limits, authentication needs, or what the output format might be. This is a significant gap for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (2 parameters, no output schema, no annotations), the description is minimally adequate but incomplete. It explains what the tool does but lacks details on behavioral aspects and usage context, which are important for an agent to operate effectively without annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, clearly documenting both parameters ('topic' and 'subtopic') with their types and purposes. The description adds no additional meaning beyond what the schema provides, so it meets the baseline of 3 where the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'contextual help about Opik Comet's capabilities', making the purpose understandable. However, it doesn't specifically differentiate from sibling tools like 'get-opik-examples' or 'get-opik-tracing-info', which also provide information about Opik Comet but focus on different aspects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as 'get-opik-examples' or 'get-opik-tracing-info'. It lacks explicit context, exclusions, or prerequisites, leaving the agent to infer usage based on the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-opik-tracing-infoCInspect
Get information about Opik's tracing capabilities and how to use them
| Name | Required | Description | Default |
|---|---|---|---|
| topic | No | Optional specific tracing topic to get information about (e.g., 'spans', 'distributed', 'multimodal', 'annotations') |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states what the tool does without disclosing behavioral traits. It doesn't cover aspects like whether it's a read-only operation, potential rate limits, authentication needs, or output format, which are critical for an agent to use it effectively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that efficiently conveys the tool's purpose without unnecessary words. It's front-loaded and appropriately sized, though it could be slightly more structured by including brief usage hints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (one optional parameter, no output schema, no annotations), the description is minimally complete but lacks depth. It explains what the tool does but doesn't provide enough context about behavior or usage relative to siblings, making it adequate but with clear gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, clearly documenting the optional 'topic' parameter with examples. The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline of 3 for adequate but not enhanced coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'information about Opik's tracing capabilities and how to use them', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get-trace-by-id' or 'get-trace-stats', which also deal with tracing information but focus on specific data rather than general capabilities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios like needing overviews versus detailed data, or how it differs from siblings such as 'get-opik-help' or 'get-opik-examples', leaving the agent to infer usage from context alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-project-by-idCInspect
Get a single project by ID
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | ID of the project to fetch | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, it doesn't specify whether this requires authentication, what happens if the project doesn't exist, or any rate limits. The description lacks crucial behavioral context that would help an agent understand how to handle errors or what to expect from the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just 6 words, front-loading the essential information with zero wasted words. Every word earns its place by communicating the core functionality without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read operation with 2 parameters and no output schema, the description is insufficiently complete. It doesn't explain what information the tool returns about projects, how to handle the optional workspaceName parameter, or what format the response takes. With no annotations and no output schema, the agent lacks crucial information about the tool's behavior and outputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents both parameters (projectId and workspaceName). The description adds no additional parameter information beyond what's in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('a single project by ID'), making the purpose immediately understandable. It distinguishes from sibling tools like 'list-projects' by specifying retrieval of a single item rather than a collection. However, it doesn't explicitly differentiate from 'get-prompt-by-id' or 'get-trace-by-id' which follow similar patterns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to prefer this over 'list-projects' for single-item retrieval, or how it relates to other 'get-by-id' tools for different resource types. There's also no information about prerequisites or contextual constraints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-prompt-by-idCInspect
Get a single prompt by ID
| Name | Required | Description | Default |
|---|---|---|---|
| promptId | Yes | ID of the prompt to fetch |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden but only states the basic action without disclosing behavioral traits. It doesn't mention if this is a read-only operation, what happens with invalid IDs (e.g., errors), authentication needs, rate limits, or return format, leaving significant gaps in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence that directly states the tool's purpose, making it front-loaded and free of unnecessary words. Every part of the sentence earns its place by clearly conveying the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a retrieval tool with no annotations and no output schema, the description is incomplete. It doesn't explain what data is returned (e.g., prompt content, metadata), error handling, or how it fits into the broader context of prompt management, failing to compensate for the lack of structured data.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning beyond the input schema, which has 100% coverage and fully documents the 'promptId' parameter. With high schema coverage, the baseline is 3, as the schema handles parameter documentation adequately without extra detail from the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'a single prompt by ID', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get-project-by-id' or 'list-prompts', which would require mentioning it retrieves individual prompt details rather than projects or lists.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as 'list-prompts' for browsing or 'get-project-by-id' for related resources. It lacks context about prerequisites like needing a valid prompt ID or exclusions for non-existent IDs, leaving usage unclear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-server-infoBInspect
Get information about the Opik server configuration
| Name | Required | Description | Default |
|---|---|---|---|
| random_string | No | Dummy parameter for no-parameter tools |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states it 'gets' information, implying a read-only operation, but doesn't disclose behavioral traits such as authentication needs, rate limits, response format, or potential side effects. This leaves significant gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It lacks details on what specific server configuration information is returned, how it's structured, or any behavioral context. For a tool in a set with many siblings, more completeness is needed to guide effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 1 parameter with 100% coverage, describing it as a 'Dummy parameter for no-parameter tools'. The description adds no parameter information, which is acceptable since the schema fully documents the parameter, and the tool effectively has 0 functional parameters. Baseline is 3, but the dummy nature justifies a higher score for clarity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'information about the Opik server configuration', making the purpose specific and understandable. However, it doesn't differentiate from sibling tools like 'get-metrics' or 'get-opik-tracing-info', which also retrieve server-related information, so it misses full sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get-metrics' and 'get-opik-tracing-info' that might retrieve overlapping or related server data, there's no indication of context, prerequisites, or exclusions for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-trace-by-idCInspect
Get a single trace by ID
| Name | Required | Description | Default |
|---|---|---|---|
| traceId | Yes | ID of the trace to fetch | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states it's a read operation ('get'), but doesn't cover aspects like error handling (e.g., what happens if the trace ID doesn't exist), rate limits, authentication requirements, or response format. For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose ('Get a single trace by ID'), making it immediately understandable. Every part of the sentence contributes directly to clarifying the tool's function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete for a tool with 2 parameters. It lacks details on behavioral traits (e.g., error cases, permissions), return values, and usage context. While the schema covers parameters well, the overall context for safe and effective use is insufficient, especially for a read operation that might involve workspace-specific data.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for both parameters: 'traceId' as the ID to fetch and 'workspaceName' as an optional override. The description adds no additional parameter semantics beyond what the schema provides, such as format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a single trace by ID' clearly states the action (get) and resource (trace), with specificity about retrieving by ID. It distinguishes from siblings like 'list-traces' (multiple) and 'get-trace-stats' (statistics), though it doesn't explicitly name them. The purpose is unambiguous but lacks explicit sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose it over 'list-traces' for multiple traces or 'get-trace-stats' for aggregated data, nor does it specify prerequisites like authentication or workspace context. Usage is implied by the name but not explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get-trace-statsCInspect
Get statistics for traces
| Name | Required | Description | Default |
|---|---|---|---|
| endDate | No | End date in ISO format (YYYY-MM-DD) | |
| projectId | No | Project ID to filter traces | |
| projectName | No | Project name to filter traces | |
| startDate | No | Start date in ISO format (YYYY-MM-DD) | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states it 'gets' statistics without disclosing behavioral traits like read-only nature, potential rate limits, authentication needs, or what happens if parameters are omitted. It fails to compensate for the lack of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, making it easy to parse quickly without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no output schema, no annotations), the description is incomplete. It doesn't explain what statistics are returned, how they're formatted, or error conditions, leaving significant gaps for the agent to infer behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no meaning beyond the schema, such as explaining how parameters interact or default behaviors. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get statistics for traces' states a clear verb ('Get') and resource ('statistics for traces'), but it's vague about what specific statistics are retrieved and doesn't distinguish from sibling tools like 'get-metrics' or 'get-trace-by-id'. It provides basic purpose but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get-metrics' or 'list-traces'. The description doesn't mention prerequisites, exclusions, or context for usage, leaving the agent without direction on tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list-projectsCInspect
Get a list of projects/workspaces
| Name | Required | Description | Default |
|---|---|---|---|
| page | Yes | Page number for pagination | |
| size | Yes | Number of items per page | |
| sortBy | No | Sort projects by this field | |
| sortOrder | No | Sort order (asc or desc) | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states it 'gets a list' which implies a read operation, but doesn't mention pagination behavior (implied by parameters), rate limits, authentication requirements, or what the return format looks like. For a tool with 5 parameters and no output schema, this leaves significant behavioral aspects undocumented.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just 5 words, front-loading the core purpose with zero wasted words. Every element ('Get', 'list', 'projects/workspaces') earns its place. No structural issues or unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what a 'project' or 'workspace' represents in this context, doesn't describe the return format, and provides no behavioral context. The agent would need to infer too much from just the schema parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter information beyond what's already in the schema descriptions. It doesn't explain relationships between parameters (e.g., how 'workspaceName' interacts with the list) or provide usage examples. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get a list') and resource ('projects/workspaces'), making the purpose immediately understandable. However, it doesn't distinguish between 'projects' and 'workspaces' or clarify if they're synonymous, and it doesn't differentiate from sibling tools like 'get-project-by-id' or 'list-prompts' which serve different but related purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'list-projects' over 'get-project-by-id' for retrieving specific projects, or 'list-prompts' for different resource types. There are no prerequisites, exclusions, or context for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list-promptsCInspect
Get a list of Opik prompts
| Name | Required | Description | Default |
|---|---|---|---|
| page | Yes | Page number for pagination | |
| size | Yes | Number of items per page |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool retrieves a list but doesn't mention pagination behavior (implied by parameters), rate limits, authentication needs, or what the return format looks like (no output schema). This leaves significant gaps for an agent to understand how the tool behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for a simple list operation, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with 2 required parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain pagination requirements, return format, or error conditions. Given the complexity (simple but with required params) and lack of structured data, more context is needed for the agent to use this tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters ('page' and 'size') clearly documented in the schema. The description adds no additional parameter semantics beyond implying list retrieval, so it meets the baseline score of 3 where the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a list of Opik prompts' clearly states the verb ('Get') and resource ('Opik prompts'), making the purpose immediately understandable. However, it doesn't distinguish this tool from its sibling 'list-projects' or 'list-traces' beyond the resource type, nor does it specify scope (e.g., all prompts vs. filtered).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get-prompt-by-id' for retrieving a specific prompt, or 'create-prompt' for adding new prompts. There's no mention of prerequisites, typical use cases, or limitations that would help an agent choose appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list-tracesCInspect
Get a list of traces
| Name | Required | Description | Default |
|---|---|---|---|
| page | Yes | Page number for pagination | |
| projectId | No | Project ID to filter traces | |
| projectName | No | Project name to filter traces | |
| size | Yes | Number of items per page | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states the action without disclosing behavioral traits. It doesn't mention if this is a read-only operation, pagination behavior, rate limits, authentication needs, or what the output looks like (e.g., list format, error handling). This leaves significant gaps for a tool with 5 parameters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It's front-loaded and efficiently conveys the core action, making it easy to parse quickly without unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no output schema, no annotations), the description is incomplete. It doesn't explain the return values, pagination implications, or how filtering parameters interact, leaving the agent with insufficient context to use the tool effectively beyond basic invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters like 'page', 'size', 'projectId' are well-documented in the schema. The description adds no additional meaning beyond 'list of traces', such as explaining how filtering works or parameter interactions, meeting the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a list of traces' states the basic action (get/list) and resource (traces), making the purpose understandable. However, it's vague about what 'traces' are (e.g., execution traces, logging traces) and doesn't distinguish from siblings like 'get-trace-by-id' or 'get-trace-stats', missing specificity for a 4-5 score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get-trace-by-id' for single traces or 'get-trace-stats' for aggregated data. The description lacks context about filtering capabilities or typical use cases, offering minimal help for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update-projectCInspect
Update a project
| Name | Required | Description | Default |
|---|---|---|---|
| description | No | New project description | |
| name | No | New project name | |
| projectId | Yes | ID of the project to update | |
| workspaceName | No | Workspace name to use instead of the default |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Update a project' implies a mutation operation but doesn't disclose behavioral traits like required permissions, whether changes are reversible, rate limits, or what happens to unspecified fields. It lacks critical context for a write operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence, 'Update a project', which is front-loaded and wastes no words. It efficiently communicates the core action, though this brevity contributes to gaps in other dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity as a mutation with 4 parameters and no annotations or output schema, the description is incomplete. It doesn't cover behavioral aspects, usage context, or return values, leaving significant gaps for an AI agent to understand and invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters (projectId, name, description, workspaceName). The description adds no meaning beyond what the schema provides, such as explaining interdependencies or default behaviors. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a project' states the verb and resource but is vague about what aspects can be updated. It distinguishes from siblings like 'create-project' and 'delete-project' by specifying 'update', but doesn't clarify scope compared to 'update-prompt' or differentiate from partial updates vs full replacements.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives is provided. The description doesn't mention prerequisites (e.g., needing an existing project), exclusions, or comparisons to siblings like 'get-project-by-id' for read operations or 'list-projects' for discovery.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update-promptCInspect
Update a prompt
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | New name for the prompt | |
| promptId | Yes | ID of the prompt to update |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure but offers none. It doesn't indicate whether this is a read-only or destructive operation, what permissions are required, whether changes are reversible, what happens on success/failure, or any rate limits. For a mutation tool with zero annotation coverage, this complete lack of behavioral information is inadequate and potentially misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is maximally concise at just three words, with zero wasted language. Every word earns its place by identifying the core action and resource. While this conciseness comes at the expense of completeness, the structure is front-loaded and efficient from a pure brevity perspective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a mutation tool with no annotations and no output schema, the description is severely incomplete. It doesn't explain what 'updating' entails beyond the name parameter, what happens to other prompt attributes, what the tool returns, or any error conditions. For a tool that modifies data, this minimal description leaves critical gaps in understanding its behavior and outcomes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with both parameters ('promptId' and 'name') clearly documented in the schema. The description adds no additional parameter information beyond what the schema already provides. According to scoring rules, when schema coverage is high (>80%), the baseline score is 3 even with no parameter information in the description, which applies here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a prompt' is a tautology that restates the tool name without adding meaningful context. While it correctly identifies the verb ('update') and resource ('prompt'), it fails to specify what aspects of a prompt can be updated or distinguish this tool from sibling tools like 'update-project'. This minimal statement provides no differentiation or specificity beyond the name itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides absolutely no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing prompt ID), exclusions, or relationships to sibling tools like 'create-prompt', 'delete-prompt', or 'get-prompt-by-id'. Without any context about appropriate usage scenarios, the agent has no basis for making informed decisions about tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
19 tool updates
v1.0.0- First observed
create-project - First observed
create-prompt - First observed
create-prompt-version - First observed
delete-project - First observed
delete-prompt - First observed
get-metrics - First observed
get-opik-examples - First observed
get-opik-help - First observed
get-opik-tracing-info - First observed
get-project-by-id - First observed
get-prompt-by-id - First observed
get-server-info - First observed
get-trace-by-id - First observed
get-trace-stats - First observed
list-projects - First observed
list-prompts - First observed
list-traces - First observed
update-project - First observed
update-prompt
TDQS
Scored across 19 tools
Each tool has a clearly distinct purpose targeting specific resources (projects, prompts, traces, metrics, help) and actions (create, get, list, update, delete). There is no overlap or ambiguity between tools, making it easy for an agent to select the right one for any operation.
All tools follow a consistent verb_noun pattern with hyphens (e.g., create-project, get-project-by-id, list-projects). The naming is uniform across all 19 tools, using clear verbs like create, get, list, update, and delete paired with specific nouns.
With 19 tools, the count is slightly high but reasonable for a comprehensive server covering projects, prompts, traces, metrics, and help. It feels well-scoped for the Opik domain, though it borders on being heavy compared to typical 3-15 tool ranges.
The tool set provides complete CRUD/lifecycle coverage for projects and prompts (create, get, list, update, delete), with additional tools for traces, metrics, and help. There are no obvious gaps; agents can perform all core operations without dead ends in this domain.
Maintenance
Related MCP Connectors
A Model Context Protocol server for Wix AI tools
Model Context Protocol server for Studex tools, notifications, and profile integrations
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to interact with yo…
Model Context Protocol server for the Apideck Unified API. Connect any MCP-compatible agent framework to 100+ accounting systems, HRIS platforms, file storage providers, and more through one integration. More information https://www.apideck.com/mcp-server
Related MCP Servers
- FlicenseBqualityDmaintenanceImplements the Model Context Protocol (MCP) to provide AI models with a standardized interface for connecting to external data sources and tools like file systems, databases, or APIs.1153-
- AlicenseNot gradedqualityCmaintenanceAn implementation of the Model Context Protocol (MCP) that enables interaction with debug adapters, allowing language models to control debuggers, set breakpoints, evaluate expressions, and navigate source code during debugging sessions.40AGPL 3.0
- FlicenseNot gradedqualityDmaintenanceA Python-based implementation of the Model Context Protocol that enables communication between a model context management server and client through a request-response architecture.-
- -licenseNot gradedqualityDmaintenanceA standardized foundation for building Model Context Protocol servers that integrate with VS Code, using Python with stdio transport for seamless AI tool integration.-