quality-kpi-mcp
Reads Argo CD deployment exports (deployed_at, first_commit_at, service, environment, status) to compute deployment frequency, lead time for changes, change failure rate, and time to restore, with DORA tier ratings and per-service change-failure hotspots.
Reads GitHub Actions deployment exports (deployed_at, first_commit_at, service, environment, status) to compute deployment frequency, lead time for changes, change failure rate, and time to restore, with DORA tier ratings and per-service change-failure hotspots.
Reads GitLab CI deployment exports (deployed_at, first_commit_at, service, environment, status) to compute deployment frequency, lead time for changes, change failure rate, and time to restore, with DORA tier ratings and per-service change-failure hotspots.
Reads Jenkins deployment exports (deployed_at, first_commit_at, service, environment, status) to compute deployment frequency, lead time for changes, change failure rate, and time to restore, with DORA tier ratings and per-service change-failure hotspots.
Reads Jira bug exports (key, priority, component, found_in, created, resolved, status) to calculate defect escape rate, reopen rate, resolution time, defect aging and SLA compliance by priority, including listing every defect past SLA. Incident exports from Jira Service Management are also supported as an incident source.
Consumes incident CSV exports (started, detected, acknowledged and resolved timestamps, severity, service, caused_by_deploy) to compute MTTR, MTTA, MTTD, customer-reported incident share, and the incident side of DORA metrics.
Consumes incident CSV exports (started, detected, acknowledged and resolved timestamps, severity, service, caused_by_deploy) to compute MTTR, MTTA, MTTD, customer-reported incident share, and the incident side of DORA metrics.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@quality-kpi-mcpWhat's our MTTR this quarter, and is it improving?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
quality-kpi-mcp
Engineering quality KPIs for leadership, on demand: MTTR, DORA metrics, defect escape rate and SLA breaches, answered by your AI assistant from the exports you already have.
quality-kpi-mcp is a Model Context Protocol server that turns incident, deployment and Jira bug exports into the numbers engineering leaders get asked for every month:
"What's our MTTR this quarter, and is it getting better?" "Where do we stand on DORA metrics?" "Which service is causing the most failed deploys?" "Write the monthly quality update for the exec team."
No BI project, no data warehouse, no spreadsheet stitched together the night before the QBR. Drop in three CSV exports, and Claude or Cursor does the analysis.

Why this exists
Every quarter I've watched engineering managers spend days assembling the same report: pull incidents from PagerDuty, deploys from CI, bugs from Jira, paste into a spreadsheet, compute MTTR by hand, then write the story. The numbers are always a week stale, and nobody trusts how they were calculated. This server makes the calculation consistent, explainable and instant.
It's the leadership companion to qa-metrics-mcp, which handles test-level triage for engineers.
Related MCP server: Koalr
What it measures
Tool | What it answers |
| What data is loaded, the date range, services, and the as-of date |
| MTTR, MTTA, MTTD: mean, median and p90, by severity, service, month or root cause, plus % of incidents customers found first |
| The four DORA keys with an Elite / High / Medium / Low tier for each and overall |
| Which services' deploys cause incidents or rollbacks, the root causes, and hotspots |
| Defect escape rate, escape rate by component, reopen rate, resolution time and SLA compliance by priority |
| Open defects by age, and every defect past SLA, worst first |
| Month-by-month series for any KPI |
| Every KPI, this period vs. last, with % change and improved / worse / flat |
| A one-page markdown report: bottom line, scorecard, DORA tiers, wins, risks and recommended asks |
| A self-contained HTML dashboard (light and dark mode) |
Plus a monthly_quality_review prompt that chains the tools into a VP-ready monthly update.
How the numbers are calculated
KPI | Definition |
MTTR | Mean time from incident start to resolved |
MTTA | Mean time from detection to first acknowledgement |
MTTD | Mean time from incident start to detection |
Deployment frequency | Production deploys per week, per service |
Lead time for changes | Median time from first commit to production deploy |
Change failure rate | Share of deploys that were rolled back or caused an incident |
Time to restore (DORA) | Median restore time for incidents caused by a deploy |
Defect escape rate | Share of defects first found in production |
SLA breach | Open defect older than its priority's SLA (Critical 2d, High 7d, Medium 30d, Low 90d) |
DORA tiers use benchmark bands based on the DORA State of DevOps reports (elite: on-demand deploys, lead time under a day, recovery under an hour). The overall tier is the weakest of the four keys. Bands and SLA policy are constants in src/metrics.ts, so you can set your own targets.
What leaders get that a dashboard doesn't
Findings, not just charts. Rule-based insights call out what changed and why it matters: "payments is a change-failure hotspot: 20% of 54 deploys failed; top cause: schema migration."
Asks, not just risks. The executive summary ends with concrete next steps an owner can take.
Any period, any comparison.
last_30_days,this_quarter,2026-09, or any date range, always compared with a sensible baseline.
Works with your existing exports
The loader reads CSV and recognizes common column names from real tools, so most exports work without editing:
File | Typical source | Key columns (aliases accepted) |
| PagerDuty, Opsgenie, incident.io, Jira Service Management |
|
| GitLab CI, GitHub Actions, Jenkins, Argo CD |
|
| Jira bug export |
|
Priorities like Highest / Blocker / P1 map to Critical, SEV1 / P1 / critical map to SEV1, and prod / live maps to production. Non-production deploys are ignored. Files are re-read on every call, so dropping in a fresh export updates every answer.
Quick start
git clone https://github.com/Misfit1982/quality-kpi-mcp.git
cd quality-kpi-mcp
npm install # also compiles TypeScript to dist/
npm run smoke # end-to-end test over real MCP stdioThe repo ships with six months of synthetic data for a fictional e-commerce company, with real stories planted in it (see Sample data).
Connect to Claude Desktop
Add to claude_desktop_config.json:
{
"mcpServers": {
"quality-kpi": {
"command": "node",
"args": ["/absolute/path/to/quality-kpi-mcp/dist/server.js"],
"env": { "KPI_DATA_DIR": "/absolute/path/to/your/exports" }
}
}
}Connect to Claude Code
claude mcp add quality-kpi -- node /absolute/path/to/quality-kpi-mcp/dist/server.jsConnect to Cursor
Add the same block to .cursor/mcp.json.
Configuration
Variable | Default | Purpose |
|
| Folder with |
|
| Where the dashboard is written |
| latest date in the data | "Today" for all calculations, e.g. |
Example session
You: Prepare the quality update for my VP. Last 90 days.
Claude: (calls
generate_executive_summary,get_change_failure_breakdown,get_defect_aging)Bottom line: Delivery is rated High on all four DORA keys. Incident recovery improved sharply: MTTR fell 63% to 4.7 hours. Two things need attention: deploy quality on payments and bugs escaping in checkout.
Wins
MTTR 12.8h → 4.7h after the on-call runbooks rollout
Customers now find 20% of incidents first, down from 43%
Lead time for changes halved to 31 hours
Risks and asks
payments: 20% of deploys failed, mostly schema migrations. Ask: require tested rollback plans for migrations.
checkout: 38% of October's defects were found in production. Ask: add regression coverage before the next release.
SHOP-2413 (gift-card orders charged full amount) has been open 43 days against a 2-day SLA. Ask: named owner and fix date this week.
Full generated output: docs/sample-executive-summary.md.
Sample data
Six months (May to October 2026) of synthetic data for a fictional company, ShopLite: about 660 production deploys, 85 incidents and 420 Jira bugs. Planted storylines the tools should find:
MTTR drops sharply after on-call runbooks roll out on Aug 1
Customer-reported incidents fall after synthetic monitoring goes live, so MTTD drops
The payments service's change failure rate spikes in September (a risky schema migration)
Defect escape rate climbs in the checkout component
A Critical checkout defect sits open for 40+ days, far past its 2-day SLA
Lead time for changes improves steadily as PRs get smaller
npm run smoke asserts that every one of them is detected.
Architecture
flowchart LR
PD[Incident tool<br/>PagerDuty · Opsgenie] -->|incidents.csv| DIR[(Exports folder)]
CI[CI/CD<br/>GitLab · Actions · Argo] -->|deployments.csv| DIR
J[Jira] -->|defects.csv| DIR
DIR --> L[data.ts<br/>CSV loader + column aliases]
L --> M[metrics.ts<br/>MTTR · DORA · defects · insights]
M --> S[server.ts<br/>10 MCP tools + review prompt]
M --> D[dashboard.ts<br/>HTML export]
S <-->|stdio / MCP| C[Claude · Cursor · any MCP client]src/
data.ts CSV parsing and normalization (severity, priority, environment, column aliases)
metrics.ts Pure KPI functions, DORA bands, SLA policy, insights, executive summary
dashboard.ts Self-contained HTML leadership dashboard
server.ts MCP server: tools, input validation (zod), error handling
scripts/
generate-sample-data.ts Deterministic synthetic data with planted storylines
smoke-test.ts End-to-end test via the official MCP clientRoadmap
Pull directly from the Jira, PagerDuty and GitLab APIs instead of CSV exports
Per-team scorecards and targets
Scheduled monthly report delivered to Slack or email
Incident-to-RCA linking with the companion RCA assistant
About
Built by Jared Hood, QA leader with 20 years in SaaS, fintech and media. I build AI-assisted QA and engineering-metrics tooling: MCP servers, test automation, and reporting that turns raw data into decisions.
All data in this repo is synthetic and describes a fictional company.
License: MIT
Available Tools
10 toolscompare_periodsCompare two periodsARead-only
Every KPI side by side for two periods with % change and improved/worse/flat. Defaults to the chosen period vs. the equal-length period before it, e.g. this quarter vs. last.
| Name | Required | Description | Default |
|---|---|---|---|
| period | No | Time window: last_30_days, last_90_days (default), this_month, last_month, this_quarter, last_quarter, year_to_date, all, YYYY-MM, or YYYY-MM-DD..YYYY-MM-DD | |
| service | No | Limit to one service/component, e.g. 'checkout' | |
| baseline | No | Baseline window (default: the equal-length period immediately before) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds real value by disclosing the output shape (side-by-side KPIs, % change, improved/worse/flat labels) and the default baseline behavior, though it says nothing about volume, pagination, or what happens with 'all'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with zero filler; the comparison semantics come first and the default behavior second. Nothing is repeated or padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so adequately by naming the output contents. For a three-optional-parameter read tool this is nearly sufficient, with only minor gaps around result volume and edge cases like the 'all' period.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter is documented in the schema, so the baseline is 3. The description's only addition is the 'e.g. this quarter vs. last' illustration of baseline derivation, which largely restates the schema's own default note.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource (every KPI side by side) and the transformation applied (% change with improved/worse/flat), which is enough to distinguish it from a single-series trend tool like get_kpi_trend. It stops short of naming any sibling explicitly, so the differentiation is inferred rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the default comparison behavior ('the chosen period vs. the equal-length period before it') and gives an example, which implies when this tool is appropriate. It never states when to prefer it over get_kpi_trend or get_dora_metrics, nor any exclusions, so usage is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
export_leadership_dashboardExport leadership dashboardA
Write a self-contained HTML dashboard (KPI tiles, wins and risks, MTTR / change failure / defect trends, DORA tiers, SLA breaches) to the output folder and return its path.
| Name | Required | Description | Default |
|---|---|---|---|
| period | No | Time window: last_30_days, last_90_days (default), this_month, last_month, this_quarter, last_quarter, year_to_date, all, YYYY-MM, or YYYY-MM-DD..YYYY-MM-DD | |
| filename | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose the write nature (readOnlyHint=false), non-destructive and closed-world scope. The description adds real context beyond them: the artifact is self-contained HTML, it lands in the output folder, and the tool returns the file path. It omits what happens on filename collision (overwrite vs error) and whether the directory is created if absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and artifact, then the output location and return value. The parenthetical list is dense but each item is a genuine dashboard section, so nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly carries the burden of describing the return value (the file path) and the artifact's contents. The remaining gap is file-handling semantics for `filename` — overwrite behavior and path resolution — which an agent would want before calling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%: `period` is fully documented in the schema with its accepted values and default, but `filename` has only a regex pattern and no prose. The description says nothing about either parameter — no default period behavior, no note on what happens if the filename is omitted or already exists — so it fails to compensate for the gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
A specific verb (Write) plus a concrete resource (a self-contained HTML dashboard) with the exact contents enumerated: KPI tiles, wins/risks, MTTR, change failure, defect trends, DORA tiers, SLA breaches. The output destination (output folder) and return value (its path) are both stated, which cleanly separates it from the read-only sibling getters and from generate_executive_summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the phrase 'leadership dashboard' and the write-to-file behavior suggest this is the reporting/export path rather than a data-retrieval call. It never states when to choose it over generate_executive_summary or compare_periods, nor any prerequisite such as data sources being configured.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
generate_executive_summaryExecutive summaryARead-only
A one-page markdown quality report for leadership: bottom line, KPI scorecard vs. the previous period, DORA tiers, wins, risks, and recommended asks. Ready for email, Slack or Confluence.
| Name | Required | Description | Default |
|---|---|---|---|
| period | No | Time window: last_30_days, last_90_days (default), this_month, last_month, this_quarter, last_quarter, year_to_date, all, YYYY-MM, or YYYY-MM-DD..YYYY-MM-DD | |
| baseline | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered; the description adds genuinely new information about the artifact produced (one-page, markdown, fixed section list) — important because no output schema exists. It omits any note on generation cost/latency or whether missing data degrades sections, which keeps it below a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler, with the deliverable and its contents front-loaded before the distribution note. Every clause carries information an agent would otherwise have to guess.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does so by enumerating report sections and format. Combined with the well-documented `period` enum, an agent has enough to invoke it correctly; the only gap is the undefined `baseline` parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The `period` parameter is fully enumerated in the schema, but `baseline` has no schema description (50% coverage). The phrase "vs. the previous period" hints at what the comparison baseline means, but the description never explains how `baseline` relates to `period` or what the default comparison is, so it only partially compensates for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the concrete deliverable — a one-page markdown quality report for leadership — and enumerates its sections (bottom line, KPI scorecard vs. prior period, DORA tiers, wins, risks, asks). That is far more specific than a restated name and lets an agent distinguish it from raw-metric siblings like get_dora_metrics or get_kpi_trend. It stops short of naming the closest sibling (export_leadership_dashboard) to draw an explicit boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"For leadership" and "Ready for email, Slack or Confluence" imply the audience and delivery context, so an agent can infer this is the polished roll-up rather than a data-fetch call. There is no explicit when-to-use/when-not guidance and no mention of when to prefer compare_periods or export_leadership_dashboard instead, leaving the routing to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_change_failure_breakdownChange failures by serviceBRead-only
Which services' deployments cause incidents or rollbacks, with rates, root causes and hotspots (services failing at 2x the overall rate).
| Name | Required | Description | Default |
|---|---|---|---|
| period | No | Time window: last_30_days, last_90_days (default), this_month, last_month, this_quarter, last_quarter, year_to_date, all, YYYY-MM, or YYYY-MM-DD..YYYY-MM-DD |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds useful analytical context by enumerating rates, root causes, and the hotspot threshold of 2x the overall rate, but it does not describe return shape, pagination, or data freshness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded phrase with no filler. Every element—services, rates, root causes, and hotspots—adds meaning without repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read-only analytical tool with full schema coverage and safety annotations, the description is largely complete: it explains what the breakdown contains and defines hotspots. It could say more about output grouping or how services are ordered, but no output schema exists to carry that burden.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single period parameter, and the schema documents all accepted formats and the default. The description adds no parameter-specific meaning beyond what the schema already provides, so the baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific analytical resource: a breakdown of which services' deployments cause incidents or rollbacks, including rates, root causes, and hotspots. It is clearly distinct from generic DORA or MTTR tools, though it does not explicitly name any sibling alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies an analytical use case but gives no explicit when-to-use guidance, prerequisites, or exclusions versus siblings like get_dora_metrics or get_mttr. It is left to the agent to infer when this breakdown is preferable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_defect_agingDefect aging & SLA breachesARead-only
Open defects right now: age buckets, open count by priority, and every defect past its SLA (Critical 2d, High 7d, Medium 30d, Low 90d), worst first.
| Name | Required | Description | Default |
|---|---|---|---|
| priority | No | ||
| component | No | Limit to one service/component, e.g. 'checkout' |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered; the description adds real value beyond that by disclosing the SLA policy thresholds, the "worst first" ordering, and that only open defects are included. It does not say whether the SLA-breach list is truncated or paginated, which matters given "every defect past its SLA" could be a large result.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler: scope, three output components, SLA thresholds, and sort order are all packed into one clause chain. Every element earns its place and nothing is repeated from the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing returns and does so adequately by naming the three result sections and the ordering. The remaining gap is bucket definition (what the age buckets actually are) and result-size behavior, which an agent would want to know before calling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%: component carries its own description and priority is a self-explanatory four-value enum, so the schema does most of the work. The description never mentions filtering at all, leaving the agent to infer from "open count by priority" that the priority parameter scopes results, which is a weak but non-zero contribution.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific resource and scope: currently open defects, broken into age buckets, counts by priority, and an SLA-breach list with the exact thresholds (Critical 2d, High 7d, Medium 30d, Low 90d). That is far more concrete than the sibling get_defect_metrics, but the description never explicitly contrasts itself with that sibling or get_mttr, so differentiation is left to inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Open defects right now" implies a point-in-time snapshot rather than a historical trend, which is a usable usage cue against trend-oriented siblings like get_kpi_trend. However, no alternative tool is named and no when-not-to-use condition is given, so the guidance remains implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_defect_metricsDefect metricsBRead-only
Defects created and resolved, escape rate (share found in production), reopen rate, escape rate by component, and resolution time and SLA compliance by priority.
| Name | Required | Description | Default |
|---|---|---|---|
| period | No | Time window: last_30_days, last_90_days (default), this_month, last_month, this_quarter, last_quarter, year_to_date, all, YYYY-MM, or YYYY-MM-DD..YYYY-MM-DD | |
| component | No | Limit to one service/component, e.g. 'checkout' |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds genuine value by defining escape rate as 'share found in production', which is a semantic clarification not available in structured fields. It says nothing about aggregation level, whether results are current or cached, or how missing components are handled — acceptable gaps given the annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with no filler, and the metric inventory is front-loaded. It reads slightly as an output dump rather than a purpose statement, but nothing in it is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of telling the agent what comes back, and the metric list does that well; the schema fully covers the two inputs. What is missing is whether numbers are point-in-time snapshots vs. trends and how they relate to get_kpi_trend or compare_periods, which keeps it short of a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%: both 'period' (with its full enum of windows plus the last_90_days default) and 'component' are documented in-schema. The description adds no parameter-level detail at all, so the baseline 3 for a fully documented schema is the right score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Enumerates the specific measures returned (created/resolved counts, escape rate, reopen rate, per-component escape rate, resolution time and SLA compliance by priority), which is materially more informative than the title 'Defect metrics'. It implicitly separates itself from siblings like get_defect_aging and get_mttr by covering breadth rather than one dimension, though it never names those alternatives explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description contains no when-to-use or when-not-to-use guidance and never references a sibling. An agent cannot tell from this text whether to call get_defect_metrics, get_defect_aging, or get_mttr when asked for 'defect health' — it has to infer from the metric list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_dora_metricsDORA metricsBRead-only
The four DORA keys (deployment frequency, lead time for changes, change failure rate, time to restore) with an Elite/High/Medium/Low tier for each and overall.
| Name | Required | Description | Default |
|---|---|---|---|
| period | No | Time window: last_30_days, last_90_days (default), this_month, last_month, this_quarter, last_quarter, year_to_date, all, YYYY-MM, or YYYY-MM-DD..YYYY-MM-DD | |
| service | No | Limit to one service/component, e.g. 'checkout' |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so safety and scope are covered. The description's added value is the return content (four keys plus tiering), which is genuinely useful given there is no output schema. It says nothing about defaults, filtering behavior, or how tiers are computed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero padding, and the most important information (the four keys) comes first. It is a fragment rather than a sentence with a verb, which slightly weakens the framing but costs no space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter, zero-required, read-only metric tool with full schema coverage and annotations, the description covers the essential question of what the caller gets back. Only the default time window and the meaning of the tier boundaries are left implicit, both minor for this complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, including the full period enumeration and the last_90_days default, so the schema carries parameter meaning on its own. The description adds nothing about period or service, which is the baseline expectation when the schema is this complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the specific resource (DORA metrics) and enumerates exactly what comes back: the four keys plus an Elite/High/Medium/Low tier per key and overall. That is far more concrete than a restated title. It stops short of distinguishing itself from siblings such as get_mttr or get_change_failure_breakdown, which cover overlapping sub-metrics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to call this tool versus the many adjacent metric tools (get_kpi_trend, compare_periods, get_change_failure_breakdown). No prerequisites, no mention of the default period, and no exclusions are offered, so the agent must infer selection from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_kpi_trendKPI trend by monthBRead-only
Month-by-month series for one KPI. Metrics: mttr_hours, mtta_minutes, mttd_minutes, incidents, customer_detected_pct, deploys_per_week_per_service, lead_time_hours, change_failure_rate_pct, defects_created, escape_rate_pct, open_defects_end_of_month.
| Name | Required | Description | Default |
|---|---|---|---|
| metric | Yes | ||
| period | No | Window to cover (default: all) | |
| service | No | Limit to one service/component, e.g. 'checkout' |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered. The description adds the meaningful trait of monthly granularity for a single metric, but says nothing about default period behavior, how far back the series goes, or what a 'series' contains.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded and short, which is good, but the entire second sentence is a verbatim copy of the schema enum — pure redundancy that occupies most of the description without adding information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description carries some burden for explaining what comes back; 'month-by-month series' gives a hint but not the shape (list of periods? values?). With only three simple params and read-only annotations, it is minimally viable but leaves gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67% and the description's metric list merely duplicates the enum already present in the schema, adding no new meaning. The 'one KPI' phrasing reinforces single-metric selection, but period and service semantics are left entirely to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: a month-by-month time series for a single KPI. The scope ('one KPI', monthly granularity) is clear, though it never names which sibling to prefer (get_mttr, get_dora_metrics) for the same underlying metrics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use / when-not-to-use guidance and no mention of alternatives, despite siblings like get_mttr and get_dora_metrics covering overlapping metrics. The agent must infer that this tool is for trending a metric over time rather than a point-in-time snapshot.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_mttrMTTR, MTTA, MTTDARead-only
Incident response metrics for a period: mean time to restore (MTTR), acknowledge (MTTA) and detect (MTTD), with median, p90 and customer-detected share. Group by severity (default), service, month or root cause.
| Name | Required | Description | Default |
|---|---|---|---|
| period | No | Time window: last_30_days, last_90_days (default), this_month, last_month, this_quarter, last_quarter, year_to_date, all, YYYY-MM, or YYYY-MM-DD..YYYY-MM-DD | |
| service | No | Limit to one service/component, e.g. 'checkout' | |
| group_by | No | ||
| severity | No | SEV1..SEV4 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds real behavioral value by disclosing what is computed (mean, median, p90, customer-detected share) and that severity is the default grouping — the latter appears nowhere in the schema. It stops short of explaining metric definitions or empty-period behavior, which keeps it below a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: the first defines the payload and its statistics, the second the grouping options. Metric names are front-loaded and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description appropriately summarizes the return content (three metric families plus medians, p90 and customer-detected share), and the schema carries period and filter formats. Only the semantics of what MTTR/MTTA/MTTD measure in this system is left unstated, a minor but real gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is a strong 75%, so the burden is light. The description still contributes the default grouping dimension ('severity (default)'), which the schema does not declare for group_by, and enumerates the same four group_by values, confirming the enum's meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the specific resource (incident response metrics), expands every acronym (MTTR/MTTA/MTTD), and enumerates the statistics returned plus the grouping dimensions. This clearly separates it from sibling metric tools like get_dora_metrics or get_defect_metrics, which cover different domains.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the metric names and the grouping sentence, but there is no explicit guidance on when to choose this over get_dora_metrics, get_kpi_trend or compare_periods, and no stated prerequisites or exclusions. An agent can infer intent from the domain, but must reason about sibling selection itself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_data_sourcesList data sourcesARead-only
What data is loaded: row counts, date range, services/components, and the as-of date used for all calculations.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered; the description adds real behavioral value by disclosing the exact payload and, notably, that the reported as-of date governs all calculations. It stops short of stating whether the listed set is fixed or reflects the caller's tenant scope.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded clause followed by an efficient colon-list of returned fields; nothing is padded. It is a fragment rather than a sentence with a verb, but it earns its length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing return values and does so item by item, including the important as-of date caveat. For a zero-parameter read-only discovery tool this is nearly sufficient; only the scope of the returned list is unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline of 4 applies; there are no argument semantics to explain. The description instead spends its words on what comes back, which is the right allocation here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description answers 'what is this' directly: it reports what data is loaded and enumerates the contents (row counts, date range, services/components, as-of date). This is a specific inventory/discovery resource rather than a metric query, which implicitly separates it from the metric siblings, though no sibling is named.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use, no prerequisite, and no reference to the metric siblings that make up the rest of the tool set. An agent can infer it is a discovery call, but the description never says to invoke it first to confirm coverage before running get_mttr, get_dora_metrics, etc.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
10 tool updates
v1.0.0- First observed
compare_periods - First observed
export_leadership_dashboard - First observed
generate_executive_summary - First observed
get_change_failure_breakdown - First observed
get_defect_aging - First observed
get_defect_metrics - First observed
get_dora_metrics - First observed
get_kpi_trend - First observed
get_mttr - First observed
list_data_sources
TDQS
Scored across 10 tools
Each tool has a clearly distinct purpose: data inventory, defect aging, defect metrics, incident MTTR, DORA metrics, change-failure breakdown, KPI trends, period comparison, executive summary, and dashboard export. Overlaps such as time-to-restore appearing in both MTTR and DORA tools are framed at different levels of aggregation, and the descriptions make the boundaries clear.
All tools use consistent snake_case with a clear verb_noun pattern: get_, list_, compare_, generate_, and export_. The naming is predictable and readable throughout, including acronym cases like get_mttr and get_dora_metrics.
Ten tools is well-scoped for a quality KPI reporting server, with separate tools for each major metric family, reporting workflow, and data discovery need. No tool appears redundant or purely decorative.
The surface covers the domain end to end: discovering loaded data, retrieving defect, incident, DORA, and change-failure metrics, trending and comparing KPIs, and producing executive-level summaries and dashboards. No obvious operational gap would block the intended reporting workflows.
Maintenance
Related MCP Connectors
Generate answers & visualizations from your engineering data to track software development health.
AI data analyst: ask about your Excel, CSV, GA4 or database data and get charts and dashboards back
Read-only AI coding tools for change verification, release readiness, capacity, and guidance.
Ask business questions in plain English. Get instant answers from your database, no SQL needed.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceEnables intelligent analysis of regression test failures and automatic discovery of solutions in JIRA. Analyzes test logs using AI-driven algorithms and matches errors with relevant JIRA issues through natural language interactions.-
- AlicenseNot gradedqualityDmaintenanceConnect engineering metrics, DORA performance, deploy risk scoring, and PR health to any AI assistant. Score PRs for deployment risk using a 36-signal model, query team health, incidents, coverage, and more.MIT
- AlicenseAqualityDmaintenanceGenerates sprint retrospectives grounded in actual data from Jira, GitHub, and Slack, providing evidence-backed insights and action items.685 npmMIT
- FlicenseNot gradedqualityBmaintenanceEnables natural-language Q&A over customer support ticket data. Provides tools for schema inspection, SQL-based ticket counts and grouping, and full-text search for customer wording without requiring API keys.-