byok-observability-mcp
byok-observability-mcp is a local MCP server for querying Grafana, Prometheus, Kafka UI, Datadog, and OpsGenie — with credentials staying on your machine.
Cross-backend
obs_health_check— Parallel health check across all configured backends with a unified status summaryobs_investigate_incident— Parallel root cause analysis: checks Grafana firing alerts, Prometheus offline endpoints, and Kafka offline brokers simultaneously
Grafana (7 tools)
Check connectivity, list datasources, execute PromQL queries, list/retrieve dashboards, list active Alertmanager alerts, and get all configured alert rules
Prometheus (5 tools)
Check connectivity, run instant and range PromQL queries, list available metric names, and retrieve metric metadata
Kafka UI (6 tools)
List clusters, topics, and consumer groups; describe topics; get per-partition consumer lag; check broker health and disk usage
Datadog
Proxied access to the official Datadog MCP server with configurable tool sets (metrics, dashboards, APM, alerting, logs, etc.)
OpsGenie (3 tools)
List open alerts, identify on-call personnel across all schedules, and acknowledge alerts
Scheduled Reports
Run
npx byok-observability-mcp --reportto send an automated observability digest to Slack with findings categorized by severity (P0 Critical → P3 Informational), schedulable via cron or macOS launchd
Proxies the official Datadog MCP server, providing tools for metrics, dashboards, monitors, APM services, traces, alerting, logs, incidents, security monitoring, and other Datadog observability features.
Provides tools for querying metrics via Grafana datasources, listing dashboards and datasources, retrieving dashboard details, and managing alerts and alert rules from Alertmanager.
Provides tools for executing PromQL queries (instant and range), listing available metrics, retrieving metric metadata, and checking Prometheus connectivity.
Bring Your Own Keys — credentials stay in env vars on your machine. No clone, no build, runs via npx.
Partial setup — configure only the backends you use. Tools for unconfigured backends are never exposed.
How it works
Related MCP server: kubernetes-mcp
⚡ Quick Start
Option A — Interactive wizard (recommended)
Run once, answer a few questions, get a ready-made .mcp.json:
npx byok-observability-mcp --initThe wizard will:
Let you pick which backends to configure
Ask for credentials per service
Test connectivity with your real endpoints before writing anything
Write
.mcp.jsonto your project root or~/.claude/— your choice
Then just start Claude Code:
claudeThat's it. No clone, no build, no env file. Works in under 60 seconds.
Option B — Manual .mcp.json
Create .mcp.json in your project root. Include only the backends you need.
{
"mcpServers": {
"observability-mcp": {
"command": "npx",
"args": ["-y", "byok-observability-mcp"],
"env": {
"GRAFANA_URL": "https://grafana.mycompany.internal",
"GRAFANA_TOKEN": "glsa_...",
"PROMETHEUS_URL": "https://prometheus.mycompany.internal",
"KAFKA_UI_URL": "https://kafka-ui.mycompany.internal",
"DD_API_KEY": "your-datadog-api-key",
"DD_APP_KEY": "your-datadog-app-key"
}
}
}
}Credentials in git? Use the
${VAR}approach instead — see Configuration → Method B.
Start Claude Code:
claudeClaude Code reads .mcp.json automatically. No claude mcp add, no build step.
Verify by asking Claude:
What observability tools do you have available?🧩 Supported clients
Client | Configuration |
Claude Code |
|
OpenAI Codex CLI |
|
Both clients read .mcp.json automatically. The Quick Start above works for either.
# Same .mcp.json as above works out of the box
codexOr add via CLI:
codex mcp add --transport stdio observability-mcp -- npx -y byok-observability-mcp🔧 Available tools
Always available. Checks connectivity across all configured backends.
Tool | Description |
| Unified Health Check. Runs a parallel check on all backends and returns a status table. |
Enabled when
GRAFANA_URL+GRAFANA_TOKENare set.
Tool | Description |
| Check connectivity, version, and database status |
| List all datasources (name, type, UID) |
| Run a PromQL expression via a Grafana datasource |
| Search and list dashboards by name or tag |
| Get panels and metadata for a dashboard by UID |
| List active alerts from Alertmanager (firing/pending) |
| List all configured alert rules across all folders |
Enabled when
PROMETHEUS_URLis set.
Tool | Description |
| Check connectivity |
| Instant PromQL query — current value of a metric |
| Range PromQL query — metric values over time |
| List all available metric names |
| Get help text and type for a specific metric |
Enabled when
KAFKA_UI_URLis set.
Tool | Description |
| List configured Kafka clusters and their status |
| List topics in a cluster |
| Get partition count, replication factor, and config |
| List consumer groups and their state |
| Get per-partition lag for a consumer group |
| Broker count and disk usage per broker |
Enabled when both
DD_API_KEYandDD_APP_KEYare set. Proxies the official Datadog MCP server.
Default toolsets: core, apm, alerting. Set DD_TOOLSETS=all to load everything.
Toolset | Covers |
| Metrics, dashboards, monitors, infrastructure |
| APM services, traces, service map |
| Monitors, downtimes, alerts |
| Log search and analytics |
| Incident management |
| SQL-style metric queries |
| Cloud security posture |
| Synthetic test results |
| Network performance monitoring |
| Database monitoring |
| CI/CD pipelines |
| LLM observability |
| Case management |
| Feature flag tracking |
🔑 Getting credentials
Open Grafana → Administration → Users and access → Service accounts
Click Add service account → set Role to
Viewer→ CreateOn the service account page → Add service account token → Generate token
Copy the token (starts with
glsa_) — you won't see it again
GRAFANA_URL=https://grafana.mycompany.internal
GRAFANA_TOKEN=glsa_xxxxxxxxxxxxxxxxIf your Grafana uses a self-signed certificate:
GRAFANA_VERIFY_SSL=falseIf Prometheus has no authentication:
PROMETHEUS_URL=https://prometheus.mycompany.internalIf Prometheus uses basic auth:
PROMETHEUS_URL=https://prometheus.mycompany.internal
PROMETHEUS_USERNAME=your-username
PROMETHEUS_PASSWORD=your-passwordIf Kafka UI has no authentication:
KAFKA_UI_URL=https://kafka-ui.mycompany.internalIf Kafka UI requires a login:
KAFKA_UI_URL=https://kafka-ui.mycompany.internal
KAFKA_UI_USERNAME=admin
KAFKA_UI_PASSWORD=your-passwordAPI key: Datadog → Organization Settings → API Keys → New Key
Application key: Datadog → Organization Settings → Application Keys → New Key
DD_SITE — match your Datadog login URL:
Login URL | DD_SITE |
|
|
|
|
|
|
|
|
|
|
DD_API_KEY=your-api-key
DD_APP_KEY=your-application-key
DD_SITE=datadoghq.com
DD_TOOLSETS=core,apm,alerting⚙️ Configuration
Method A — Values directly in .mcp.json (simplest)
Put credentials directly in .mcp.json. Works everywhere, no extra steps.
Add .mcp.json to your .gitignore if the repo is shared.
Use ${VAR} placeholders in .mcp.json and put real values in .env.
.mcp.json (safe to commit — contains no secrets):
{
"mcpServers": {
"observability-mcp": {
"command": "npx",
"args": ["-y", "byok-observability-mcp"],
"env": {
"GRAFANA_URL": "${GRAFANA_URL}",
"GRAFANA_TOKEN": "${GRAFANA_TOKEN}",
"PROMETHEUS_URL": "${PROMETHEUS_URL}",
"KAFKA_UI_URL": "${KAFKA_UI_URL}",
"DD_API_KEY": "${DD_API_KEY}",
"DD_APP_KEY": "${DD_APP_KEY}"
}
}
}
}.env (add to .gitignore):
GRAFANA_URL=https://grafana.mycompany.internal
GRAFANA_TOKEN=glsa_...Start Claude with the env loaded:
set -a && source .env && set +a && claudeA ready-made helper script is included:
./scripts/run-claude-with-env.shA template .mcp.json with all variables is available as .mcp.json.example.
Add to ~/.claude.json:
{
"mcpServers": {
"observability-mcp": {
"command": "npx",
"args": ["-y", "byok-observability-mcp"],
"env": {
"GRAFANA_URL": "https://grafana.mycompany.internal",
"GRAFANA_TOKEN": "glsa_..."
}
}
}
}📋 Environment variables
Variable | Backend | Required | Description |
| Grafana | ✅ | Base URL of your Grafana instance |
| Grafana | ✅ | Service account token (Viewer role) |
| Grafana | Set to | |
| Prometheus | ✅ | Base URL of your Prometheus instance |
| Prometheus | Basic auth username | |
| Prometheus | Basic auth password | |
| Kafka UI | ✅ | Base URL of your Kafka UI instance |
| Kafka UI | Login username | |
| Kafka UI | Login password | |
| Datadog | ✅ | Datadog API key |
| Datadog | ✅ | Datadog Application key |
| Datadog | Datadog site (default: | |
| Datadog | Tool groups to load (default: | |
| Reports | ✅* | Slack Incoming Webhook URL for scheduled reports |
| Reports | Comma-separated backends to include in reports (default: all configured) |
📊 Scheduled Reports
Send an automated observability digest to Slack on a schedule — no Claude or Codex instance needs to be running.
How it works
cron / launchd
│ fires every N minutes
▼
npx byok-observability-mcp --report
│
│ reads env vars, connects directly to backends
▼
Grafana · Prometheus · Kafka UI
│
│ categorizes findings → P0 / P1 / P2 / P3
▼
Slack Incoming Webhook → #your-channelThe command collects data, categorizes every finding by severity, formats a Slack message, sends it, and exits. It is completely stateless.
Severity levels
Level | Meaning | Examples |
🔴 P0 — KRİTİK | Service down or unreachable | Grafana alert firing (critical), Kafka cluster offline, backend unreachable |
🟠 P1 — YÜKSEK | Degraded, action needed soon | Grafana alert firing (non-critical), Kafka consumer lag > 10 000 |
🟡 P2 — ORTA | Warning, monitor closely | Grafana alert pending, Kafka consumer lag > 1 000 |
🟢 P3 — BİLGİ | Informational, all normal | Healthy backends, silenced alerts |
Setup
Step 1 — Get a Slack Incoming Webhook URL
Go to api.slack.com/apps → Create New App → From scratch
Incoming Webhooks → toggle on → Add New Webhook to Workspace
Pick a channel → Allow → copy the Webhook URL
Step 2 — Set environment variables
export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/XXX/YYY/ZZZ
# Optional: restrict which backends are included (default: all configured)
export REPORT_BACKENDS=grafana,prometheus,kafkaStep 3 — Run a one-off report to verify
npx byok-observability-mcp --reportYou should see a message in your Slack channel within seconds.
Step 4 — Schedule with cron
Open your crontab:
crontab -eAdd a line. Examples:
# Every hour at minute 0
0 * * * * SLACK_WEBHOOK_URL=https://hooks.slack.com/... GRAFANA_URL=... GRAFANA_TOKEN=... npx byok-observability-mcp --report >> /tmp/obs-report.log 2>&1
# Every 30 minutes
*/30 * * * * SLACK_WEBHOOK_URL=https://hooks.slack.com/... npx byok-observability-mcp --report >> /tmp/obs-report.log 2>&1Put all env vars in a.env file and source it inside the cron command to keep the crontab clean:
0 * * * * bash -c 'source /path/to/.env && npx byok-observability-mcp --report' >> /tmp/obs-report.log 2>&1Alternative: macOS launchd (runs on login, survives reboots)
Create ~/Library/LaunchAgents/com.observability-mcp.report.plist:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.observability-mcp.report</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/npx</string>
<string>byok-observability-mcp</string>
<string>--report</string>
</array>
<key>EnvironmentVariables</key>
<dict>
<key>SLACK_WEBHOOK_URL</key>
<string>https://hooks.slack.com/services/XXX/YYY/ZZZ</string>
<key>GRAFANA_URL</key>
<string>https://grafana.mycompany.internal</string>
<key>GRAFANA_TOKEN</key>
<string>glsa_...</string>
</dict>
<key>StartInterval</key>
<integer>3600</integer>
<key>StandardOutPath</key>
<string>/tmp/obs-report.log</string>
<key>StandardErrorPath</key>
<string>/tmp/obs-report.log</string>
</dict>
</plist>Load it:
launchctl load ~/Library/LaunchAgents/com.observability-mcp.report.plistTo stop: launchctl unload ~/Library/LaunchAgents/com.observability-mcp.report.plist
💬 Example prompts
Single-backend queries
Backend | Try asking Claude... |
Grafana | "List all datasources and tell me which ones are Prometheus type." |
Grafana | "Search for dashboards related to 'kubernetes' — list names and UIDs." |
Grafana | "Query |
Prometheus | "What is the current value of the |
Prometheus | "Show CPU usage ( |
Prometheus | "List all available metrics that start with |
Kafka UI | "List all Kafka clusters. Are there any with offline brokers?" |
Kafka UI | "Describe the topic 'orders' in cluster 'production' — partitions and replication factor?" |
Kafka UI | "Check consumer lag for group 'order-processor'. Which partitions have the highest lag?" |
Datadog | "List all Datadog monitors currently in Alert state." |
Datadog | "Show APM service performance for the past hour. Which services have the highest error rate?" |
Datadog | "Query |
🛠️ Incident Response (v0.2.0+)
Goal | Try asking Claude... |
Health | "Run a health check on all systems." |
Alerts | "Are there any firing alerts in Grafana right now?" |
Triage | "Show me the alert rules for the 'Production' folder." |
Cross-backend queries
Check the health of all configured observability backends and give me a summary.I'm seeing high error rates. Check Prometheus for http_requests_total with status=500,
then look for related Datadog monitors that might be alerting.🔒 Security
All tools areread-only. No write operations are performed on any backend.
Credentials are read from environment variables andnever logged or sent to Anthropic. Tokens are redacted in all error messages.
TLS certificate verification is enabled by default
The MCP process runs locally — your infrastructure URLs only reach Claude's context window if you type them into the chat
Least-privilege recommendations:
Backend | Recommended role |
Grafana | Service account with Viewer role |
Prometheus | Network-level read-only access |
Kafka UI | Read-only UI user |
Datadog | API key + Application key with read scopes |
🛠 Development
git clone https://github.com/alimuratkuslu/byok-observability-mcp
cd byok-observability-mcp
npm install
npm run dev # run with tsx (no build step)
npm run build # compile to dist/
npm run typecheck # TypeScript check without emittingTested versions
Backend | Tested version |
Grafana | v9.x, v10.x, v11.x |
Prometheus | v2.x |
Kafka UI |
|
License
MIT
Available Tools
23 toolsgrafana_get_alert_rulesA
List all configured Grafana alert rules from the provisioning API. Returns rule UID, title, condition, labels, annotations, folder, and rule group. Use this to see what alert rules are defined.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses the return fields (rule UID, title, etc.) and implies a read-only operation, but does not mention behavioral aspects like permissions needed, rate limits, pagination, or error handling. It adds some context but lacks comprehensive behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero waste: the first states the action and resource, the second provides usage context. It is front-loaded with the core purpose and efficiently structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and 0 parameters, the description is adequate but has gaps. It explains the purpose and output fields, but does not cover behavioral aspects like authentication requirements or response format details. For a tool with no structured data, it provides a basic but incomplete picture.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately does not discuss parameters, focusing instead on purpose and output. This meets the baseline for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('all configured Grafana alert rules from the provisioning API'), specifying the source (provisioning API) and scope (all). It distinguishes from siblings like 'grafana_list_alerts' by focusing on rules rather than active alerts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use it ('to see what alert rules are defined'), but does not explicitly state when not to use it or name alternatives like 'grafana_list_alerts' for active alerts. This gives good guidance but lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
grafana_get_dashboardA
Get full details of a Grafana dashboard including all panels, by its UID. Use grafana_list_dashboards to find the UID.
| Name | Required | Description | Default |
|---|---|---|---|
| uid | Yes | The UID of the dashboard. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes what the tool returns ('full details including all panels'), which is helpful behavioral context. However, it doesn't disclose other behavioral aspects like authentication requirements, error handling, rate limits, or response format details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. The first sentence states the purpose and parameter requirement, the second provides essential usage guidance. Every word earns its place and information is front-loaded appropriately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read operation with no output schema, the description provides good context about what the tool returns and how to obtain the required parameter. However, without annotations or output schema, it could benefit from more detail about the response structure or error conditions, though the essential information is present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the single parameter 'uid' well-documented in the schema. The description adds minimal value beyond the schema by mentioning the UID is needed, but doesn't provide additional context about UID format, validation, or examples. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Get full details'), the resource ('Grafana dashboard including all panels'), and the required identifier ('by its UID'). It distinguishes from the sibling tool grafana_list_dashboards by specifying that tool is needed to find the UID first.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool ('Get full details... by its UID') and provides a clear alternative for finding the UID ('Use grafana_list_dashboards to find the UID'). This gives complete guidance on the workflow between these two tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
grafana_healthA
Check Grafana connectivity and retrieve version and database status. Use this to verify the Grafana integration is working.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions what the tool does (check connectivity, retrieve status) but lacks details on behavioral traits such as whether it requires authentication, potential rate limits, error handling, or what the output format looks like. For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves operationally.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise and well-structured, consisting of two sentences that directly state the purpose and usage guidelines. Every sentence earns its place by providing essential information without any fluff or redundancy, making it easy to parse and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but not fully complete. It explains what the tool does and when to use it, but without annotations or an output schema, it lacks details on behavioral aspects like authentication needs or return format. For a health-check tool, this is a minimal viable description that covers basics but leaves room for improvement in transparency.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and the schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, and it appropriately doesn't mention any. This meets the baseline for tools with no parameters, as it avoids unnecessary information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Check Grafana connectivity and retrieve version and database status.' This specifies the verb ('check', 'retrieve') and resources ('connectivity', 'version', 'database status'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'prometheus_health' or 'obs_health_check', which serve similar health-check functions for other systems.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: 'Use this to verify the Grafana integration is working.' This gives a specific use case (verification of integration) that guides the agent appropriately. It doesn't explicitly state when not to use it or name alternatives (e.g., other health checks for different systems), but the context is sufficient for basic guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
grafana_list_alertsA
List active (or filtered) alerts from Grafana Alertmanager. Supports filtering by state and label selectors. Returns alert name, state, severity, labels, annotations, and start time. Use this to answer 'are there any firing alerts right now?'
| Name | Required | Description | Default |
|---|---|---|---|
| state | No | Filter alerts by state. 'firing' = active, unsuppressed alerts. 'pending' = alerts in evaluation, not yet firing. 'resolved' = inactive alerts. Omit to get all alerts. | |
| labels | No | Comma-separated label matchers to filter alerts, e.g. 'team=backend,env=prod'. Each matcher is passed as a separate Alertmanager filter param. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It successfully describes what the tool returns (alert name, state, severity, labels, annotations, and start time) and mentions filtering capabilities. However, it doesn't disclose potential limitations like rate limits, authentication requirements, or pagination behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise with two sentences that each serve distinct purposes: the first explains what the tool does and what it returns, the second provides explicit usage guidance. There's zero wasted verbiage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only listing tool with no annotations and no output schema, the description does well by specifying what data is returned and providing a clear use case. However, it could be more complete by mentioning whether this shows all alerts or only recent ones, or if there are any limitations on the data returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents both parameters. The description mentions filtering by state and label selectors but doesn't add any semantic information beyond what's in the schema descriptions. This meets the baseline expectation when schema coverage is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('active (or filtered) alerts from Grafana Alertmanager'), and distinguishes it from siblings by specifying it's for Grafana alerts rather than Opsgenie alerts or other Grafana resources like dashboards or datasources. The example use case further clarifies its specific purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'Use this to answer "are there any firing alerts right now?"' This provides clear context for its primary use case. It also implies filtering capabilities that differentiate it from other alert-related tools like opsgenie_list_alerts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
grafana_list_dashboardsA
List dashboards in Grafana with an optional search query. Returns UID, title, folder, and tags.
| Name | Required | Description | Default |
|---|---|---|---|
| query | No | Optional search string to filter dashboards by title. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the return format (UID, title, folder, and tags) and that it's a list operation, which implies read-only behavior. However, it doesn't mention potential limitations like pagination, rate limits, authentication needs, or error conditions, leaving gaps in behavioral context for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with zero waste: the first states the action and optional feature, the second specifies the return data. It's front-loaded with the core purpose, and every word contributes essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description provides basic completeness by stating the purpose, parameter use, and return fields. However, for a list operation with potential complexity (e.g., large result sets, filtering nuances), it lacks details on behavior like pagination, sorting, or error handling, making it adequate but with clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the schema fully documenting the optional 'query' parameter. The description adds minimal value beyond the schema by mentioning 'optional search query' and that it filters 'by title,' which is already in the schema description. This meets the baseline for high schema coverage without significant enhancement.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'List dashboards in Grafana' specifies the verb (list) and resource (dashboards). It distinguishes from siblings like grafana_get_dashboard (fetch specific dashboard) and grafana_list_alerts (list alerts instead), but doesn't explicitly contrast with all siblings. The optional search query adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by mentioning 'optional search query' for filtering, suggesting it's for browsing or searching dashboards. However, it lacks explicit guidance on when to use this versus alternatives like grafana_get_dashboard (for detailed view) or grafana_list_datasources (for other resources). No when-not-to-use scenarios or prerequisites are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
grafana_list_datasourcesA
List all datasources configured in Grafana (name, type, UID). Use this to find the UID needed for grafana_query_metrics.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It implies a read-only operation by using 'List' and mentions the output format, but lacks details on permissions, rate limits, or error handling. It adds some context about the UID's use, but behavioral traits are not fully disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the purpose and includes usage guidance without any wasted words. Every part of the sentence adds value, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is mostly complete for a list operation. It explains the purpose, usage, and output details, but could improve by mentioning behavioral aspects like permissions or limitations, though this is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so no parameter information is needed. The description does not add param details, which is appropriate, earning a baseline score of 4 as it compensates adequately for the lack of parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('List all datasources'), the resource ('configured in Grafana'), and the output details ('name, type, UID'). It distinguishes from siblings by focusing on datasources rather than alerts, dashboards, or metrics, making the purpose explicit and distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use this tool ('to find the UID needed for grafana_query_metrics') and provides a clear alternative (grafana_query_metrics), guiding the agent on its purpose and relationship with other tools in the context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
grafana_query_metricsA
Execute a PromQL expression via a Grafana Prometheus datasource and return the results. Use grafana_list_datasources first to find the datasource UID.
| Name | Required | Description | Default |
|---|---|---|---|
| datasource_uid | Yes | The UID of the Prometheus datasource in Grafana. | |
| expr | Yes | The PromQL expression to evaluate. | |
| from | No | Start of the time range, e.g. 'now-1h', 'now-30m'. Defaults to 'now-1h'. | |
| to | No | End of the time range, e.g. 'now'. Defaults to 'now'. | |
| instant | No | If true, returns a single instant value. If false, returns a time series. Defaults to true. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions the need to obtain a datasource UID first (a prerequisite), but doesn't disclose other behavioral aspects like authentication requirements, rate limits, error handling, or what format the results will be returned in. The description adds some context but leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise with just two sentences. The first sentence states the core functionality, and the second provides essential usage guidance. Every word earns its place with zero redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description should provide more behavioral context for a query execution tool. While it covers purpose and usage guidelines well, it doesn't describe what format results will be returned in, error conditions, or authentication requirements. For a tool with 5 parameters and no structured output documentation, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents all 5 parameters. The description doesn't add any parameter-specific information beyond what's in the schema. According to guidelines, when schema coverage is high (>80%), the baseline score is 3 even with no param info in description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Execute a PromQL expression'), target resource ('via a Grafana Prometheus datasource'), and outcome ('return the results'). It distinguishes itself from sibling tools like grafana_list_datasources by focusing on query execution rather than listing resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly provides when-to-use guidance by stating 'Use grafana_list_datasources first to find the datasource UID.' This clearly directs users to a prerequisite tool and establishes a workflow, distinguishing it from alternatives like prometheus_query which might work directly with Prometheus.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kafka_broker_healthC
Get broker health for a Kafka cluster — broker IDs, hosts, ports, and disk usage.
| Name | Required | Description | Default |
|---|---|---|---|
| cluster_name | Yes | The Kafka cluster name. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only operation by using 'Get', but does not specify if it requires authentication, has rate limits, or details the response format (e.g., JSON structure, error handling). This leaves significant gaps for a tool with potential operational impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the purpose and key details. It avoids redundancy and wastes no words, though it could be slightly more structured by separating usage context from data output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (health check with one parameter), no annotations, and no output schema, the description is minimally adequate. It covers what data is returned but lacks details on behavior, error cases, or output format, which are important for a health monitoring tool in a system like Kafka.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, clearly documenting the single required parameter 'cluster_name'. The description does not add any parameter-specific details beyond what the schema provides, such as format examples or constraints, so it meets the baseline for high schema coverage without extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get broker health') and resource ('for a Kafka cluster'), and specifies the data returned (broker IDs, hosts, ports, disk usage). However, it does not explicitly differentiate from sibling tools like 'kafka_list_clusters' or 'kafka_describe_topic', which reduces it from a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, such as needing the cluster name, or compare it to related tools like 'kafka_list_clusters' for listing clusters or 'kafka_describe_topic' for topic details, leaving usage context unclear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kafka_consumer_group_lagA
Get consumer lag for a specific consumer group — per-partition offset, end offset, and lag. Highlights partitions with non-zero lag.
| Name | Required | Description | Default |
|---|---|---|---|
| cluster_name | Yes | The Kafka cluster name. | |
| group_id | Yes | The consumer group ID. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior by specifying the data returned (offsets and lag) and a key feature (highlighting non-zero lag partitions). However, it does not cover aspects like rate limits, authentication needs, or error conditions, leaving some gaps for a tool that likely queries a monitoring system.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that efficiently conveys the tool's purpose and key behavior without unnecessary words. It is front-loaded with the main action and includes essential details, making it highly concise and effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is reasonably complete. It covers what the tool does and a behavioral trait, but lacks details on output format, error handling, or dependencies. With no output schema, more information on return values would be beneficial, but it is sufficient for basic understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, clearly documenting both parameters ('cluster_name' and 'group_id'). The description does not add any additional meaning or context beyond what the schema provides, such as format examples or constraints. Baseline score of 3 is appropriate as the schema handles parameter documentation adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('consumer lag for a specific consumer group'), with specific details about what data is retrieved ('per-partition offset, end offset, and lag') and a behavioral trait ('Highlights partitions with non-zero lag'). It distinguishes itself from siblings like 'kafka_list_consumer_groups' by focusing on detailed lag metrics rather than listing groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing detailed lag metrics for a specific consumer group, but does not explicitly state when to use this tool versus alternatives like 'kafka_list_consumer_groups' or 'kafka_describe_topic'. It provides context but lacks explicit guidance on exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kafka_describe_topicC
Get detailed information about a specific Kafka topic including partition layout, replication, and segment info.
| Name | Required | Description | Default |
|---|---|---|---|
| cluster_name | Yes | The Kafka cluster name. | |
| topic_name | Yes | The topic name. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. While it indicates this is a read operation ('Get'), it doesn't disclose important behavioral traits like authentication requirements, rate limits, error conditions, response format, or whether this operation has side effects. The description is minimal and lacks operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that communicates the core purpose without unnecessary words. It's appropriately sized for a simple read operation and front-loads the key information. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what the return value looks like, what format the information comes in, or how to interpret the results. Given that this is a read operation with 100% schema coverage, the description should provide more context about the output and operational considerations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents both parameters (cluster_name and topic_name). The description doesn't add any parameter-specific information beyond what's in the schema. It mentions 'specific Kafka topic' which aligns with the topic_name parameter but provides no additional semantic context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'detailed information about a specific Kafka topic', with specific details about what information is included (partition layout, replication, segment info). It distinguishes from sibling tools like kafka_list_topics by focusing on detailed information for a specific topic rather than listing topics. However, it doesn't explicitly contrast with all possible siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when this tool is appropriate versus kafka_list_topics (which lists topics) or other Kafka tools. There's no context about prerequisites, error conditions, or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kafka_list_clustersA
List all Kafka clusters configured in Kafka UI. Returns name, status, broker count, topic count, and partition info.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the return data but does not cover critical aspects like whether this is a read-only operation, potential rate limits, authentication requirements, error handling, or data freshness. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core action and resource, followed by specific return details. Every word contributes to understanding the tool's function without redundancy or unnecessary elaboration, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the purpose and output fields but misses behavioral context like safety, performance, or error conditions. For a simple read operation, this is minimally viable but lacks depth for robust agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so no parameter information is needed. The description appropriately focuses on output semantics, listing the returned fields. This exceeds the baseline of 3 by adding value beyond the empty schema, though it does not fully compensate for the lack of output schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List all Kafka clusters') and resource ('configured in Kafka UI'), distinguishing it from sibling tools like kafka_list_topics or kafka_list_consumer_groups. It specifies the scope ('all') and the return data fields (name, status, broker count, topic count, partition info), making the purpose explicit and specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as kafka_broker_health or other Kafka-related siblings. It lacks context about prerequisites, exclusions, or specific scenarios where this tool is preferred, leaving the agent to infer usage based on the purpose alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kafka_list_consumer_groupsC
List consumer groups in a Kafka cluster with their state and member count.
| Name | Required | Description | Default |
|---|---|---|---|
| cluster_name | Yes | The Kafka cluster name. | |
| page | No | Page number for pagination. Defaults to 1. | |
| per_page | No | Number of groups per page. Defaults to 50. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool lists groups with state and member count, but lacks details on permissions needed, rate limits, pagination behavior beyond schema hints, error handling, or what the output looks like. This is a significant gap for a tool with potential operational impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes useful details without redundancy. Every word earns its place, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete for a tool with three parameters and operational context. It lacks details on behavioral traits, output format, error conditions, and usage context, which are critical for an AI agent to invoke it correctly in a production environment.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all three parameters. The description adds no additional parameter semantics beyond implying the tool operates on a Kafka cluster, which is already covered by the cluster_name parameter. Baseline 3 is appropriate as the schema handles the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('consumer groups in a Kafka cluster') with additional details about what information is included ('their state and member count'). It's specific but doesn't explicitly differentiate from sibling tools like kafka_list_clusters or kafka_list_topics, though the resource focus is distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites, context, or exclusions, such as when to use kafka_consumer_group_lag for more detailed group analysis or how it relates to other Kafka tools in the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kafka_list_topicsA
List topics in a Kafka cluster with partition count, replication factor, and under-replicated partition count.
| Name | Required | Description | Default |
|---|---|---|---|
| cluster_name | Yes | The Kafka cluster name. Use kafka_list_clusters to see available clusters. | |
| page | No | Page number for pagination. Defaults to 1. | |
| per_page | No | Number of topics per page. Defaults to 50. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates this is a read operation ('List') and specifies what data is returned. However, it doesn't mention important behavioral aspects like whether this requires authentication, rate limits, error conditions, or pagination behavior beyond what's in the schema. The description adds value by specifying the return data format but leaves gaps in operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that contains no wasted words. It front-loads the core purpose and includes all necessary information about what data is returned. Every element of the description serves a clear purpose in helping an agent understand what the tool does.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only listing tool with no annotations and no output schema, the description provides good coverage of what the tool does and what data it returns. It specifies the exact metrics included in the listing, which is valuable context. However, without annotations or output schema, it could benefit from more behavioral context about authentication, error handling, or response format details. The description is mostly complete but has minor gaps given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents all three parameters. The description doesn't add any parameter-specific information beyond what's in the schema descriptions. It mentions the resource being operated on (topics) which aligns with the cluster_name parameter, but provides no additional semantic context about parameters. Baseline 3 is appropriate when the schema does all the parameter documentation work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('topics in a Kafka cluster') with specific details about what information is included ('partition count, replication factor, and under-replicated partition count'). It distinguishes from siblings like kafka_describe_topic (which likely provides detailed topic metadata) and kafka_list_clusters (which lists clusters rather than topics).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description doesn't explicitly state when NOT to use this tool, but it provides clear context for usage (listing topics with specific metrics). The input schema's description for cluster_name references kafka_list_clusters as an alternative for discovering available clusters, giving some guidance on prerequisites. However, it lacks explicit comparisons with other topic-related tools like kafka_describe_topic.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
obs_health_checkA
Run a health check across all configured observability backends (Grafana, Prometheus, Kafka UI, Datadog) in parallel and return a status summary table. Use this to answer 'are all systems up?'
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it runs checks 'in parallel' and returns a 'status summary table'. However, it lacks details on error handling, timeout behavior, or authentication requirements, which would be helpful for a tool interacting with multiple backends.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and scope in the first sentence, followed by a clear usage guideline. Every sentence adds value without redundancy, making it efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (checking multiple backends) and lack of annotations/output schema, the description is mostly complete. It explains what the tool does and when to use it, but could benefit from mentioning the format of the 'status summary table' or any prerequisites for backend configuration.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately focuses on the tool's behavior rather than parameters, earning a high baseline score for not introducing unnecessary complexity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Run a health check') and the resources involved ('all configured observability backends: Grafana, Prometheus, Kafka UI, Datadog'), distinguishing it from sibling tools that target individual systems. It explicitly answers the question 'are all systems up?' which reinforces its distinct purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use this to answer 'are all systems up?''. This clearly indicates when to use this tool (for overall system status) versus sibling tools that focus on specific components like grafana_health or prometheus_health.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
obs_investigate_incidentA
Meta-tool that performs parallel root cause analysis (RCA) queries across all enabled backends. Automatically checks Grafana for firing alerts, Prometheus for offline endpoints (up==0), and Kafka clusters for offline brokers.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: it's a 'meta-tool' that performs 'parallel' queries across multiple backends, automatically checking specific systems for predefined issues (alerts, offline endpoints, offline brokers). However, it doesn't mention potential limitations like rate limits, authentication requirements, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise and well-structured in two sentences. The first sentence establishes the tool's nature as a meta-tool and its purpose. The second sentence provides specific implementation details about which backends it checks and for what conditions. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity as a meta-tool with no parameters and no output schema, the description provides good contextual completeness. It explains what the tool does, which systems it queries, and what it looks for. However, without an output schema, it doesn't describe what format the RCA results will be returned in, which is a minor gap for a tool performing complex multi-system analysis.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description appropriately doesn't discuss parameters since none exist, maintaining focus on the tool's functionality rather than unnecessary parameter explanations.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as a 'meta-tool that performs parallel root cause analysis (RCA) queries across all enabled backends.' It specifies the exact actions (checks Grafana for firing alerts, Prometheus for offline endpoints, and Kafka for offline brokers) and distinguishes itself from sibling tools by being a comprehensive investigation tool rather than individual backend queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly indicates when to use this tool: for 'root cause analysis (RCA)' investigations. It implies alternatives by listing specific backend checks (Grafana, Prometheus, Kafka), suggesting that individual sibling tools like grafana_list_alerts or prometheus_query could be used for targeted queries instead of this comprehensive meta-tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
opsgenie_ack_alertC
Acknowledge an open OpsGenie alert to prevent escalations.
| Name | Required | Description | Default |
|---|---|---|---|
| alert_id | Yes | The full ID of the alert to acknowledge. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool acknowledges alerts to prevent escalations, which hints at a mutation action, but lacks details on permissions, side effects, error handling, or response format. This is insufficient for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and every part earns its place, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a mutation tool with no annotations and no output schema, the description is incomplete. It lacks information on behavioral traits, return values, error conditions, and how it fits with sibling tools, making it inadequate for the agent to fully understand the tool's context and usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the parameter 'alert_id' clearly documented. The description does not add any additional meaning beyond the schema, such as format examples or constraints, so it meets the baseline for high schema coverage without compensating value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Acknowledge') and resource ('an open OpsGenie alert'), with the purpose 'to prevent escalations' adding useful context. However, it does not explicitly differentiate from sibling tools like 'opsgenie_list_alerts' or 'obs_investigate_incident', which might handle related alert operations, so it falls short of a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for 'an open OpsGenie alert' but provides no explicit guidance on when to use this tool versus alternatives, such as other alert-handling tools in the sibling list. There are no prerequisites, exclusions, or comparisons mentioned, leaving the agent with minimal direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
opsgenie_list_alertsC
List currently open OpsGenie alerts. Returns ID, tinyId, message, priority, and acknowledged status.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Max number of alerts to return (default 20, max 100). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool lists 'currently open' alerts and returns specific fields, but lacks details on permissions, rate limits, pagination (beyond the 'limit' parameter), or error handling. This is a significant gap for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, stating the core purpose in the first sentence and output details in the second. Both sentences earn their place by providing essential information without redundancy. However, it could be slightly improved by integrating usage context or behavioral notes more seamlessly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (one parameter, no output schema, no annotations), the description is minimally adequate. It covers what the tool does and what it returns, but lacks behavioral context (e.g., safety, limitations) and usage guidelines. For a simple read operation, this is acceptable but leaves clear gaps that could hinder effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the 'limit' parameter well-documented in the schema. The description does not add any parameter-specific information beyond what the schema provides, such as default values or constraints. With high schema coverage, the baseline score of 3 is appropriate, as the description does not compensate but also does not detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'List currently open OpsGenie alerts' specifies the verb (list) and resource (alerts), and 'Returns ID, tinyId, message, priority, and acknowledged status' indicates the output scope. However, it does not explicitly differentiate from sibling tools like 'grafana_list_alerts', which might list alerts from a different system, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools (e.g., 'opsgenie_ack_alert' for handling alerts or 'grafana_list_alerts' for a different system) or any context for selection. Usage is implied only by the tool name and description, with no explicit when/when-not instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
opsgenie_who_is_on_callB
List current on-call participants for all schedules. Helps you find who to page.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the output ('list current on-call participants') and a use case ('find who to page'), but fails to describe critical behaviors like response format, pagination, error handling, authentication needs, or rate limits. For a tool with zero annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise and front-loaded, consisting of two short sentences that directly state the tool's function and utility. Every word earns its place, with no redundant or vague phrasing, making it efficient for an AI agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't explain the return format (e.g., list structure, fields like names or schedules), error scenarios, or operational constraints. For a tool that likely returns critical on-call data, more context is needed to ensure reliable use by an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the inputs. The description adds no parameter information, which is acceptable here since no parameters exist. A baseline of 4 is appropriate as it doesn't need to compensate for any gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'List current on-call participants for all schedules.' It specifies the verb ('List'), resource ('on-call participants'), and scope ('for all schedules'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'opsgenie_list_alerts' or 'opsgenie_ack_alert', which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance with 'Helps you find who to page,' suggesting it's for identifying contacts during incidents. However, it lacks explicit when-to-use rules, alternatives (e.g., vs. 'opsgenie_list_alerts' for alert details), or prerequisites, leaving some ambiguity for an AI agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
prometheus_healthB
Check Prometheus connectivity. Returns healthy/unhealthy status.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool checks connectivity and returns a status, but lacks details on authentication needs, rate limits, error handling, or what 'healthy/unhealthy' entails (e.g., response format, thresholds). This is a significant gap for a health-check tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise and front-loaded, consisting of two short sentences that directly state the action and outcome with zero wasted words. Every sentence earns its place by providing essential information efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is minimally adequate. It explains what the tool does but lacks context on behavioral aspects like authentication or error handling. For a simple health-check tool, this is acceptable but leaves clear gaps in completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100% (since there are no parameters to describe). The description doesn't need to add parameter semantics, so it meets the baseline of 4 for tools with no parameters, as it doesn't have to compensate for any gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Check Prometheus connectivity' with the outcome 'Returns healthy/unhealthy status.' It uses a specific verb ('Check') and resource ('Prometheus connectivity'), though it doesn't explicitly distinguish it from sibling tools like 'grafana_health' or 'obs_health_check' beyond the Prometheus focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools (e.g., 'grafana_health' for Grafana or 'obs_health_check' for OBS), prerequisites, or specific contexts for usage, leaving the agent to infer based on the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
prometheus_list_metricsB
List all available metric names in Prometheus. Useful for discovery when you don't know the exact metric name.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions the tool is 'useful for discovery,' which hints at a read-only operation, but fails to disclose critical behavioral traits such as whether it requires authentication, rate limits, pagination, or the format of returned data. This leaves significant gaps for an agent to use it effectively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose in the first sentence, followed by a brief usage note. Both sentences earn their place by adding value, and there is no wasted text, making it highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't address behavioral aspects like authentication needs, rate limits, or return format, which are crucial for a tool interacting with a system like Prometheus. The purpose is clear, but operational context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately adds no parameter details, maintaining focus on the tool's purpose without redundancy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('all available metric names in Prometheus'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'prometheus_metric_metadata' or 'prometheus_query', which could have overlapping discovery functions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance with 'Useful for discovery when you don't know the exact metric name,' suggesting when to use it. However, it lacks explicit alternatives (e.g., vs. 'prometheus_metric_metadata' for metadata) or exclusions, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
prometheus_metric_metadataA
Get help text, type, and unit for a specific Prometheus metric. Omit metric_name to list all metadata.
| Name | Required | Description | Default |
|---|---|---|---|
| metric_name | No | The metric name to look up. Omit to retrieve metadata for all metrics. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's dual behavior (specific metric vs. all metrics) but lacks details on permissions, rate limits, error handling, or response format. It adequately covers basic operation but misses deeper behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It front-loads the core purpose and immediately follows with the key usage note, making it easy to parse and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is complete enough for basic use. However, it lacks details on output structure (e.g., format of returned metadata) and error cases, which could be helpful for an agent invoking it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents the parameter. The description adds value by explaining the semantic effect of omitting the parameter ('Omit metric_name to list all metadata'), which clarifies usage beyond the schema's technical specification.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('Get help text, type, and unit') and resource ('for a specific Prometheus metric'), and distinguishes it from siblings by focusing on metadata retrieval rather than listing metrics (prometheus_list_metrics) or querying values (prometheus_query).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool vs. alternatives: 'Omit metric_name to list all metadata' indicates the tool's dual functionality, and the context of sibling tools (e.g., prometheus_list_metrics for listing metric names without metadata) clarifies its role in the ecosystem.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
prometheus_queryA
Execute an instant PromQL query and return the current value(s). Best for checking current state of a metric (e.g. CPU usage right now).
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | The PromQL expression to evaluate. | |
| time | No | Evaluation timestamp as RFC3339 or Unix timestamp. Defaults to current time. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explains this is an instant query (not a range query) and returns current values, which is useful context. However, it doesn't mention authentication requirements, rate limits, error behavior, or what format the results come in (though the lack of output schema means this information would be helpful).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise with two sentences that each earn their place. The first sentence states the core functionality, and the second provides crucial usage guidance with a helpful example. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a query tool with 2 parameters (100% schema coverage) but no annotations and no output schema, the description provides adequate purpose and usage guidance. However, it lacks information about authentication, error handling, or result format that would be helpful given the absence of output schema and annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents both parameters. The description doesn't add any additional parameter semantics beyond what's in the schema descriptions. The baseline of 3 is appropriate when the schema does all the parameter documentation work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Execute an instant PromQL query'), the resource ('current value(s)'), and distinguishes it from sibling tools by specifying it's for 'current state' checking rather than range queries or other Prometheus operations like health checks or listing metrics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool ('Best for checking current state of a metric') and provides a concrete example ('e.g. CPU usage right now'), which implicitly distinguishes it from the sibling 'prometheus_query_range' tool that would be used for time-range queries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
prometheus_query_rangeA
Execute a PromQL range query and return a time series. Use this to see how a metric changed over time.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | The PromQL expression to evaluate. | |
| start | No | Start time as RFC3339 or Unix timestamp. Defaults to 1 hour ago. | |
| end | No | End time as RFC3339 or Unix timestamp. Defaults to now. | |
| step | No | Query resolution step, e.g. '60s', '5m', '1h'. Defaults to '60s'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool executes a query and returns a time series, but lacks details on authentication requirements, rate limits, error handling, or what the returned time series structure looks like. For a query tool with no annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise with two sentences that each serve a distinct purpose: the first states what the tool does, and the second provides usage guidance. There's zero wasted language, and it's front-loaded with the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (query execution with temporal parameters), lack of annotations, and no output schema, the description is minimally adequate. It covers the basic purpose and usage context but lacks details on authentication, error handling, and return format that would be helpful for an AI agent. The high schema coverage helps compensate somewhat.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, providing clear documentation for all 4 parameters. The description doesn't add any parameter-specific information beyond what's in the schema (e.g., it doesn't explain PromQL syntax or provide examples). With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Execute a PromQL range query and return a time series.' It specifies the verb ('Execute'), resource ('PromQL range query'), and outcome ('return a time series'). However, it doesn't explicitly differentiate from its sibling 'prometheus_query' (which likely executes instant queries), though the mention of 'range query' and 'see how a metric changed over time' provides some implicit distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: 'Use this to see how a metric changed over time.' This indicates it's for temporal analysis rather than single-point queries. However, it doesn't explicitly mention when not to use it or name alternatives (like 'prometheus_query' for instant queries), which would be needed for a perfect score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
23 tool updates
v0.1.0- First observed
grafana_get_alert_rules - First observed
grafana_get_dashboard - First observed
grafana_health - First observed
grafana_list_alerts - First observed
grafana_list_dashboards - First observed
grafana_list_datasources - First observed
grafana_query_metrics - First observed
kafka_broker_health - First observed
kafka_consumer_group_lag - First observed
kafka_describe_topic - First observed
kafka_list_clusters - First observed
kafka_list_consumer_groups - First observed
kafka_list_topics - First observed
obs_health_check - First observed
obs_investigate_incident - First observed
opsgenie_ack_alert - First observed
opsgenie_list_alerts - First observed
opsgenie_who_is_on_call - First observed
prometheus_health - First observed
prometheus_list_metrics - First observed
prometheus_metric_metadata - First observed
prometheus_query - First observed
prometheus_query_range
TDQS
Most tools are clearly distinct by backend (Grafana, Kafka, OpsGenie, Prometheus) and action, with minimal overlap. However, some potential confusion exists between grafana_list_alerts (active alerts) and grafana_get_alert_rules (configured rules), and between prometheus_query (instant) and prometheus_query_range (range), though descriptions help clarify.
Tool names follow a highly consistent backend_action_resource pattern (e.g., grafana_list_dashboards, kafka_describe_topic, prometheus_query_range). All use snake_case, and verbs like 'list', 'get', 'query' are applied predictably across backends, making the set easy to navigate.
With 23 tools, the count is on the high side for a single server, potentially overwhelming. However, given the broad observability domain covering multiple backends (Grafana, Kafka, OpsGenie, Prometheus), it's borderline but justifiable as each tool serves a specific purpose.
The toolset provides comprehensive read/query capabilities across all backends, with good coverage for monitoring, alerting, and incident investigation. Minor gaps include lack of write/update tools (e.g., creating dashboards or alerts) and limited OpsGenie actions beyond listing and acknowledging, but core workflows are well-supported.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
An MCP server giving access to Grafana dashboards, data and more.
The Buildkite MCP server exposes Buildkite product data (pipelines, builds, jobs, and test data) to AI tools, editors, and agents through the Model Context Protocol. It provides capabilities including pipeline creation and management, build monitoring with specialized tools like 'wait_for_build', efficient log querying using Apache Parquet conversion and caching, and OAuth-based authentication for both read-write and read-only access to Buildkite's REST API.
The Cortex MCP server provides read-only access to real-time engineering context from the Cortex developer portal, allowing AI coding assistants to answer natural language questions about your organization's catalog (microservices, libraries, domains, teams, infrastructure), scorecards (engineering standards and best practices), initiatives (goals and deadlines), and Engineering Intelligence metrics. It includes tools for querying documentation, tracking personal entities, and accessing AI-assisted insights across the entire Cortex ecosystem.
MCP server for secureFlows: token-free URL builders and integration-linting tools for AI agents.
Related MCP Servers
FlicenseBqualityFmaintenanceA Model Context Protocol server that lets AI assistants interact with the Sentry API to retrieve and analyze error data, manage projects, and monitor application performance.1111-- AlicenseNot gradedqualityCmaintenanceA Model Context Protocol (MCP) server that provides safe, read-only access to Kubernetes resources for debugging and inspection. Built with security in mind, it offers comprehensive cluster visibility without modification capabilities.44MIT
- AlicenseBqualityDmaintenanceEnables interaction with Datadog's monitoring and observability platform through the MCP protocol. Supports incident management, monitor status checks, log searches, metrics queries, APM traces, dashboard access, RUM analytics, host management, and downtime scheduling.1321Apache 2.0
- AlicenseNot gradedqualityCmaintenanceAn MCP server that enables querying logs and metrics from Graylog, Prometheus, and InfluxDB 2.x. It provides tools for executing Lucene log searches, PromQL queries, and Flux queries directly within MCP-compatible clients.MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/alimuratkuslu/byok-observability-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server