Skip to main content
Glama
MatiasLaukka

Support Ticket Triage MCP

by MatiasLaukka

Knotty

Evidence-grounded support-ticket triage, diagnostics, and human-governed AI workflows.

Knotty began as a personal experiment with AI agent Skills and Model Context Protocol (MCP) servers. Connecting an assistant to a few local tools raised harder questions: How can an agent reason about incomplete evidence? What happens when an investigation changes? Who can authorize a fix? Can earlier decisions be replayed after a restart?

What began as a tool-integration exercise evolved into a local diagnostic and workflow engine—and, more recently, an evaluation platform for hybrid retrieval, AI reasoning, and reliability engineering.

Knotty's central question is:

How do we build AI systems that do more than generate plausible answers—systems whose decisions are evidence-grounded, traceable, testable, and safe to act on?

Knotty is an experimental engineering project, not a deployed customer-support platform. The local application operates on synthetic tickets and clean-room knowledge articles from Northstar Marketing Cloud, a fictional ecommerce marketing platform. There is no live help-desk connection or outgoing customer email. Separately, the research harness evaluates historical public Home Assistant issues and documentation—not live Home Assistant tickets.

Try the demo · Understand the architecture · Explore Hybrid research · Read the evaluation evidence · Detailed engineering reference

Quick start

Run the local Approval Desk to see a ticket move through classification, evidence gathering, human approval, diagnosis, verification, and closure.

Requirements: Node.js ^20.19.0, ^22.12.0 or >=24.0.0; npm; and a local clone. The commands below use PowerShell, but the npm commands work from other shells.

npm ci
npm run build

# Stop other Knotty runtime/MCP processes before resetting.
npm run reset:demo
npm run approval-desk

Open the URL printed by the server (normally http://127.0.0.1:5177). Use TKT-1010 first: a vague report becomes an evidence-based product-performance investigation after a customer reply. The local demonstration can add a controlled synthetic reply after you approve an outbound update. For an incident-routing example, try TKT-1001.

npm run reset:demo is destructive to local demo runtime state. Stop all MCP and Approval Desk processes first. The reset restores both the authoritative operational state and advisory learning state.

For narrower resets, use npm run reset:operational-demo (tickets and workflow history) or npm run reset:learning-demo (advisory learning data). To retain changes across restarts, stop the Approval Desk with Ctrl+C and rerun npm run approval-desk without a reset. The reset controller uses leases, validation, rollback, and backup checks; its complete behavior is documented in the engineering reference.

Knotty Approval Desk overview showing local synthetic tickets and workflow controls

The browser interface provides the conversation timeline, current recommendation, evidence checklist, human-review controls, and audit context. Other views: conversation workspace and recommendation panel.

Related MCP server: ToolBridge

What Knotty can do today

Capability

Status and boundary

Local MCP server, Codex Skill, browser Approval Desk

Implemented for synthetic support tickets

Conversation-aware classification and evidence checks

Implemented; deterministic safety policy has final authority

Diagnostic lifecycle, diagnosis review, fixes, verification and closure

Implemented with explicit human gates

Optional GPT classification, diagnosis advice and response drafting

Advisory, contract-checked, with local fallback

Knowledge discovery, operator promotion and controlled reuse

Implemented with a separate advisory learning ledger

B3 semantic/lexical retrieval, B4 ranking, B5 applicability

Research or shadow paths; no automatic operational cutover

H1–H5b Hybrid reasoning

Latest shadow architecture, not fully integrated into the operational diagnosis path

SIMPLE vs. HYBRID / Qwen vs. Luna vs. Sol

Historical Home Assistant experiment ongoing; no diagnostic-accuracy winner

Live Zendesk/Jira/email integration, production identity boundary

Not included

The application supports a ticket lifecycle rather than a one-shot classifier. A customer can provide new evidence, changing the classification, missing-evidence requirements, plausible diagnosis, or next permitted action. A failed fix or later reply can invalidate an earlier review. Neither a good-looking model explanation nor a retrieved article automatically proves a root cause.

Architecture in one picture

Knotty architecture overview with human/agent interfaces, shared deterministic service, transactional operational state, governed knowledge and isolated Hybrid research

Open the system architecture at full size

One shared TriageService is authoritative. The browser UI and MCP tools are adapters to the same application rules—not separate sources of truth. Four boundaries organize the system:

  1. Operational: deterministic policy, approval checks, ticket lifecycle, causal events, and transactional state.

  2. Advisory AI: optional model classification signals and drafts; outputs must pass validators and cannot authorize changes.

  3. Governed learning: reviewed knowledge candidates become usable only through explicit promotion and approved evidence policies.

  4. Research/shadow: retrieval, applicability and Hybrid reasoning experiments record observations without changing authoritative tickets.

How a recommendation is produced

Knotty triage and recommendation flow, from untrusted ticket text and evidence to validated draft, human decision and audited local action

Open the recommendation diagram at full size

  • Input remains untrusted. Ticket text, customer replies, and embedded instructions are evidence—not policy or approval.

  • Deterministic policy wins. Security, outage, SLA, evidence-readiness, revision, and escalation rules cannot be overridden by an LLM.

  • Drafting is bounded. Optional model-generated customer text must satisfy local response contracts; unavailable or invalid provider output falls back to deterministic drafting.

  • Nothing is automatically sent. Approving a customer-response field records a local workflow event; the project has no external messaging integration.

The full classifier, drafting and safety details are in the technical reference and threat model.

Human approval and the ticket lifecycle

Human-governed ticket triage sequence: investigate, save a pending proposal, require explicit approval, validate and commit, then read back

Open the approval sequence at full size

A proposal is not a ticket change. The operator first reviews evidence, risks, citations, proposed fields, and the customer-facing response. The service checks explicit actor/field authorization, source revision, routing, and lifecycle state before committing approved mutations. Readback confirms what changed and what stayed unchanged.

The lifecycle extends past this single approval:

evaluate → collect evidence → record diagnosis → human diagnosis review → customer update → fix or escalation → verification → ready-to-close → explicit closure

Exact actions vary by evidence and state. The system can escalate unresolved ambiguity instead of manufacturing a definitive answer. Fix approval is distinct from closure; new evidence can require review again.

The Approval Desk's Workflow Bar manages the current action, and the separate Pattern Bar appears for governed knowledge discovery. Their blocking reviews are enforced through the same backend service.

For a chronological MCP replay without network calls:

npm run evaluate:lifecycle-replay

Detailed diagnosis-and-fix walkthrough · Lifecycle replay guide · Full MCP action contracts

Hybrid diagnostic reasoning

Hybrid is the latest architectural expansion, but it must not be mistaken for an operationally deployed replacement for the existing SIMPLE workflow.

Knotty Hybrid diagnostic research architecture showing time-eligible historical evidence, B3/B4/B5 retrieval research, SIMPLE and HYBRID model lanes, blinded human review and no operational cutover

Open the Hybrid architecture at full size

The research architecture keeps the following pieces separate:

Research component

What is implemented or being investigated

B3 discovery

Local-first lexical retrieval, optional semantic embeddings, and provenance-preserving candidate sets across articles, known causes, playbooks and eligible resolved cases

B4 ranking

Offline lexical/semantic/reciprocal-rank-fusion comparisons; development evidence was inconclusive, so no ranking policy was promoted to authority

B5 applicability

Research on whether evidence actually applies to a case, including abstention and negative controls

H1–H5b Hybrid reasoning

Frozen, causally eligible inputs; provider-neutral contracts; immutable shadow runs; replay, idempotency and safe failure handling

Operational integration

Not completed; shadow output cannot approve, diagnose authoritatively, fix or close tickets

The purpose is not merely to supply more context to an LLM. It's to ask whether combining retrieval, structured evidence, diagnostic taxonomy, and model reasoning produces better-supported hypotheses without encouraging unsupported certainty or leaking knowledge from the future.

The experiments include an existing SIMPLE baseline and a HYBRID shadow lane. Provider/model identity is kept separate from the case opportunity and the human review packet. A claim that Hybrid works better requires scored, comparable results—not just successful execution.

Hybrid shadow runner design · B3 retrieval plan · B4 ranking · B5 applicability · Hybrid implementation PR #30

Home Assistant historical-case experiment (E1a)

Knotty's earlier demonstrations use fictional support tickets. The E1a research goes further by studying historical public Home Assistant GitHub issues and time-eligible documentation. This is a controlled, local historical-case experiment, not an integration with Home Assistant support or an endorsed benchmark.

Every stage is limited by an asOf cutoff. A model should not see later GitHub comments, future solutions, or current documentation substituted for a missing historical version. Eligible documentation must have exact historical provenance and content hashes; otherwise it stays unavailable.

Preliminary technical results — October 2026

Measure

Status

Public historical cases

24

Time-scoped evidence checkpoints

64

SIMPLE/HYBRID × Qwen/Luna/Sol attempts

384

Structurally valid provider outputs

341

Invalid provider outputs

43

Valid-output packets prepared for blinded review

341

Completed diagnostic-accuracy comparison

Not yet available

341/384 is an output-validity result, not diagnosis accuracy. Malformed outputs remain reliability failures but do not receive invented human semantic ratings. Model and architecture identities remain concealed during blinded review; no provider ranking, Hybrid superiority claim, or winner is justified while adjudication and analysis are unfinished.

The E1a harness and raw evaluation artifacts currently reside in a separate local experimental worktree rather than in a reproducible benchmark command on this repository's main branch. Earlier local Qwen semantic-retrieval metrics and other controlled GPT comparisons are different experiments.

Governed knowledge evolution

Knotty governed knowledge evolution: completed diagnoses, candidate discovery, human promotion, immutable versions, advisory learning ledger and future evidence-gated reuse

Open the knowledge evolution diagram at full size

Knotty can suggest reusable knowledge after supported diagnoses, but this is not autonomous model retraining: no model can promote its own candidates or convert a discovered pattern into policy.

The sequence is: observed evidence → completed diagnosis → deterministic candidate discovery → optional advisory drafting → human policy review → explicit promotion → versioned future reuse. Candidates with incomplete or undecided evidence policies stay blocked. An approved known cause can still require new evidence in a later ticket; rejected, stale and contradicted objects do not silently become operational truth.

The SQLite learning ledger tracks candidate maturity and health, explicit promotions, immutable versions, reuse events and contradictions. Previously evaluated tickets retain their original source versions, so historical recommendations remain unchanged even as the approved knowledge catalog evolves.

npm run demo:knowledge-evolution
npm run demo:learning-ledger
npm run evaluate:knowledge-holdout

Detailed evidence-policy rules · Learning and operational failure boundaries

Persistence and reliability

Knotty reliability architecture with command IDs, deterministic policy validation, atomic operational SQLite, isolated learning outbox and restart-safe readback

Open the persistence and reliability diagram at full size

The authoritative operational plane uses transactional SQLite, not a set of mutable JSON/JSONL ticket records. Governed commands commit the projection/revision, ticket-local causal event, safe trace, replay result and any learning-outbox intent together.

Problem

Design choice

Stale human decisions

Optimistic source-revision and approval-time checks

Duplicate clicks or lost responses

Stable command IDs and persistent idempotent results

Ordering and auditability

Causal ticket-local event sequences and safe read models

Crash/restart uncertainty

Replayed command receipts; no reapplication of successful mutation

Learning repository unavailable

Operational success is isolated; advisory delivery retries or reports unavailability

Demo reset during a running process

Usage leases, guarded reset, verification and rollback

These are local reliability guarantees. SQLite files can still be modified by someone with filesystem access; this is neither a distributed multi-tenant help desk nor a cryptographically tamper-evident audit ledger.

npm run demo:operational-persistence
npm run evaluate:lifecycle-replay

For actual migration states, import contracts, reset safeguards, and crash/recovery restrictions, use the engineering reference.

Using the MCP server and Codex Skill

The repository-local Skill teaches Codex to inspect the current workflow, gather evidence, propose the next action and pause at the approval boundary.

  1. Run npm ci and npm run build in the repository root.

  2. Open the root as a project in Codex desktop. Review and trust .codex/config.toml; it starts node dist/src/index.js using the project working directory.

  3. Start a new thread and invoke the triaging-support-tickets Skill.

Use $triaging-support-tickets to review TKT-1005.
Show evidence, citations, escalation risks and proposed fields.
Stop and wait for my explicit approval before changing anything.

The standalone npm start script runs the stdio MCP server, not an HTTP page. An idle terminal is expected while it waits for MCP messages.

The server exposes 27 typed tools, 4 resources and 3 prompts (at the documented implementation baseline). Useful starting points include list_tickets, get_ticket_workflow, search_knowledge, evaluate_ticket, record_diagnosis, review_diagnosis, mark_response_done, and get_audit_events. Every mutation has specific schema and policy constraints; use the complete read, mutation, resource and prompt reference.

Optional model assistance

By default, the Approval Desk uses deterministic response drafting and needs no API key. Explicit opt-in example:

$env:OPENAI_API_KEY = 'set-locally-do-not-commit'
$env:APPROVAL_DRAFT_PROVIDER = 'openai'
npm run approval-desk

Provider availability is not required for deterministic routing, lifecycle, audit, or approval checks. Don't put credentials in tickets, screenshots or committed configs.

Useful configuration defaults

Environment setting

Default

TRIAGE_DATA_ROOT

data/runtime

TRIAGE_SEED_FILE

data/seed/tickets.json

TRIAGE_KNOWLEDGE_ROOT

data/knowledge

OPERATIONAL_DB_PATH

data/runtime/operational.sqlite

TRIAGE_LEARNING_LEDGER_PATH

data/runtime/knowledge-evolution/learning.sqlite

TRIAGE_KNOWLEDGE_APPROVERS

support-lead,reviewer,approval-desk

TRIAGE_MINUTES_SAVED

8 (an assumption, not measured labor time)

Advanced environment/provider options · Safe reset guidance · Retrieval index controls

Evaluation and current results

Different harnesses answer different questions. Fixture agreement, local controlled simulation, provider-output validity, and live-model diagnostic quality are not interchangeable metrics.

Command

What it proves (and what it does not)

npm test

Build, typecheck and local Vitest regression tests

npm run evaluate

Agreement between 30 preconstructed recommendations and expected outcomes; not unseen-ticket accuracy

npm run evaluate:diagnostics

11 controlled scenarios across 8 diagnostic families; nonmutating checks

npm run evaluate:lifecycle

Deterministic audit across all 30 seeded tickets; not 30 full chronological replays

npm run evaluate:lifecycle-replay

Real MCP lifecycle and read-before-action gates using temporary state

npm run evaluate:knowledge-holdout

Controlled future-turn evidence and approved-knowledge reuse

npm run evaluate:ai-comparison

Controlled local deterministic/advisory classifier and draft lanes; --live is explicitly opt-in

npm run evaluate:ai-diagnosis

Article-backed diagnoses and controlled optional advisory providers

npm run evaluate:retrieval

Local retrieval metrics; optional separately configured semantic experiments

npm run evaluate:ranking / npm run evaluate:applicability

B4/B5 development research, not an authorized operational ranking policy

To run the broad local verification (no live-model calls by default):

npm ci
npm test
npm run verify:portfolio

The 30-ticket seeded evaluate sample is intentionally constructed to match its reference outcomes; a printed perfect accuracy from that sample is not evidence of real-world diagnostic performance. Likewise, older GPT response-quality figures represent specific dated controlled or authenticated runs with separately documented limitations. E1a human evaluation is still in progress.

Full metrics, historical scorecards and commands · Diagnostic harness methods · AI comparison example · Retrieval reports

Trust boundaries and limitations

  • Synthetic local application: No live customer data, messaging, Zendesk/Jira updates, paging, or identity-provider integration. Home Assistant issues are historical external research inputs only.

  • Prompt injection: Ticket text and retrieved content cannot authorize actions, rewrite policy, or approve fields.

  • Explicit human intent: Pending proposals need review; an MCP payload alone cannot establish that a human personally intended an approval.

  • Causal time: Historical research must exclude future comments and documentation versions unavailable by the case cutoff.

  • Provider uncertainty: Malformed, contradictory, or unsupported model output can fail, abstain or fall back. Research gains are not assumed.

  • Audit and persistence: Local SQLite transactions are durable within documented conditions but aren't distributed, authenticated, or tamper-proof.

  • Reported savings: Queue estimatedMinutesSaved uses a configured assumption per approved recommendation, not stopwatch measurements.

  • Fixed fixtures: Synthetic SLA deadlines use June 10, 2026; use historical asOf where time dependence matters.

See SECURITY.md for the full threat model and limitations in the engineering reference.

Repository guide

Where to look

Purpose

src/triage-service.ts

Authoritative governed commands

src/policy.ts

Escalation, validation and field rules

src/server.ts

MCP tools, resources, prompts and error boundaries

src/approval-desk/

Browser workflow, classification and drafting

src/operational/

Transactional SQLite, event projections and replay

src/knowledge-evolution/

Candidate governance, approved reuse and learning ledger

src/retrieval/

Lexical/semantic retrieval, ranking, applicability

src/reasoning/

Frozen-input Hybrid shadow reasoning

data/seed/ and data/knowledge/

Local synthetic fixtures and articles

test/ and scripts/

Regression tests, demos and evaluators

.agents/skills/triaging-support-tickets/

Codex Skill and policy guidance

Documentation: Engineering reference · Case study · Demo walkthrough · Knowledge evolution · Lifecycle viewer · Evaluation guide · Roadmap · Capture guide.


Knotty is an ongoing personal engineering project. The implemented system is built around human oversight, deterministic policy and reproducible evidence; the Hybrid research asks how far that architecture can be improved without losing those boundaries.

Available Tools

9 tools
approve_triage_recommendationC
Destructive

Apply only explicitly approved recommendation fields to the ticket.

ParametersJSON Schema
NameRequiredDescriptionDefault
actorYes
confirmYes
ticketIdYes
approvedFieldsYes
expectedRevisionYes
recommendationIdYes
editedCustomerResponseNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
ticketYes
auditEventYes

TDQS

C2.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and readOnlyHint=false, so the agent knows this is a destructive write. The description adds the important nuance that only explicitly approved fields are applied, implying unapproved fields remain unchanged. However, it does not disclose that this overwrites ticket fields, that confirm=true is required, or that expectedRevision provides optimistic locking.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words, which is concise. However, for a tool with 7 parameters, destructive behavior, and optimistic locking, this is under-specified rather than appropriately concise. It could be expanded without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex: it involves a destructive update, a required confirmation flag, an expectedRevision for concurrency, and 7 parameters. Yet the description provides only one line and does not cover the flow, prerequisites, or consequences. Even with annotations and schema, the description leaves significant gaps for safe usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for explaining parameters. It vaguely references 'recommendation fields' which maps to approvedFields, but does not explain recommendationId, ticketId, expectedRevision, actor, or the confirm mechanism. This leaves critical parameter behavior undocumented at the description level.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Apply only explicitly approved recommendation fields to the ticket.' The verb 'apply', the resource 'ticket', and the qualifier 'only explicitly approved fields' distinguish it from sibling tools like submit_triage_recommendation and reject_triage_recommendation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives, nor any mention of prerequisites like having a pending recommendation or needing to confirm approval. The sibling tool names imply context, but the description itself provides no explicit when/when-not direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_similar_ticketsA
Read-onlyIdempotent

Find likely duplicate tickets using deterministic text similarity.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
candidatesYes
sourceTicketIdYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds valuable behavioral information about the algorithm being 'deterministic' and the results being only 'likely' duplicates, which clarifies that the tool is heuristic and reproducible, beyond what annotations state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that is front-loaded with the tool's purpose and method. It contains no unnecessary words or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (1 param, output schema present, annotations clear) and the description covers the main purpose. However, it omits explanation of the 'id' parameter and does not offer usage guidance, leaving some gaps for an agent to infer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description mentions no parameters. The schema defines 'id' as a required string with a pattern, but the description doesn't explain that 'id' refers to the ticket to find similar ones. Schema coverage is 0%, and the description fails to compensate by describing the parameter's meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action: 'Find likely duplicate tickets' using a defined method ('deterministic text similarity'). This distinguishes it from sibling tools like list_tickets or get_ticket, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a use case (finding duplicate tickets) but provides no explicit guidance on when to choose this tool over alternatives such as search_knowledge or get_ticket. The uniqueness of the duplicate-detection purpose is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_audit_eventsB
Read-onlyIdempotent

Read all audit events or events for one ticket.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
offsetNo
ticketIdNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
limitYes
totalYes
eventsYes
offsetYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the scoping to one ticket but does not disclose pagination behavior, ordering, or what happens when ticketId is omitted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no unnecessary words. It efficiently communicates the core functionality.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema and safety annotations, so basic use is covered. However, the description is minimal for a paginated read tool with an optional filter; it does not explain pagination or the precise behavior when ticketId is missing, leaving some gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description must compensate. It only hints at ticketId via 'one ticket' and does not explain limit or offset semantics. The parameter names are somewhat self-explanatory, but the description lacks explicit mapping and context for all three parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies a read operation for audit events, with explicit scope: all events or events for one ticket. This distinguishes it from sibling tools like get_ticket or list_tickets, which operate on tickets, not audit events.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. There is no mention of audit trail use cases, prerequisites, or exclusions, leaving the agent without explicit decision support.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_queue_metricsA
Read-onlyIdempotent

Calculate current queue, SLA, recommendation, and savings metrics.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
generatedAtYes
openTicketsYes
rejectionRateYes
ticketsByTeamYes
acceptanceRateYes
escalationCountsYes
slaAtRiskTicketsYes
untriagedTicketsYes
averageConfidenceYes
ticketsByCategoryYes
ticketsByPriorityYes
slaBreachedTicketsYes
estimatedMinutesSavedYes
pendingRecommendationsYes
approvedRecommendationsYes
rejectedRecommendationsYes
submittedRecommendationsYes
minutesPerAcceptedRecommendationYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the metric categories and the 'current' qualifier, providing useful context about the tool's scope without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler, stating exactly what the tool calculates. It is appropriately sized for a zero-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the zero-parameter input, comprehensive annotations, and presence of an output schema, the description sufficiently explains the tool's purpose and scope. It names all metric types returned, making it complete without over-explaining.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline score of 4 applies. The description does not need to explain any parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Calculate' and identifies four distinct metric categories (queue, SLA, recommendation, savings). This clearly differentiates it from sibling tools that focus on individual tickets, audit events, or recommendation approvals.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit usage guidance or alternative tools are mentioned. The description implies it's the metrics aggregation tool, but doesn't state when to prefer it over list_tickets or get_ticket, so usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_ticketA
Read-onlyIdempotent

Read one support ticket by ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
ticketYes

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safe read behavior is covered. The description adds no new behavioral context beyond what is already known, such as error handling or permissions. It does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is direct and efficient. Every word serves a purpose, with no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-one-ticket tool with an output schema and rich annotations, the description is sufficient. The tool's behavior is fully captured by 'read one support ticket by ID', and no additional context about return values or side effects is necessary given the structured metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description carries the parameter explanation burden. It says 'by ID', which clarifies that the 'id' property is the ticket identifier, but it does not elaborate on the expected format beyond the schema's pattern (TKT-####). The single parameter is straightforward, so this is adequate but not enriching.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Read one support ticket by ID' uses a specific verb ('read') and resource ('support ticket') with the identifier method ('by ID'). It clearly distinguishes from siblings like list_tickets and search_knowledge, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you have a specific ticket ID and need to fetch a single ticket, which is a reasonable inference. However, it does not explicitly contrast with alternatives like list_tickets or find_similar_tickets, nor does it mention when not to use it (e.g., for bulk retrieval).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_ticketsB
Read-onlyIdempotent

Filter and page through the local support ticket queue.

ParametersJSON Schema
NameRequiredDescriptionDefault
asOfNo
teamNo
limitNo
offsetNo
statusNo
categoryNo
priorityNo
slaStateNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
itemsYes
limitYes
totalYes
offsetYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only and non-destructive, so the description need not repeat that. It adds the 'local' qualifier and pagination hint, but lacks deeper behavioral context like default sort order or handling of missing filters. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no wasted words. It is concise and direct, though somewhat under-specified given the tool's parameter count.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 optional parameters and a rich set of filter enums, the description is too brief. It gives only a high-level summary and does not cover the variety of filtering dimensions (status, priority, team, etc.). An output schema exists, which reduces the need to describe return values, but the description still lacks essential context for a complex listing tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain any of the 8 parameters. It generically says 'filter' which hints at the filter parameters, but does not add meaning beyond their raw names and enums. The description fails to compensate for the lack of schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's verb ('filter and page') and resource ('local support ticket queue'), distinguishing it from siblings like get_ticket (which fetches a single ticket). However, it does not explicitly name a sibling for contrast, so it falls just short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for browsing ticket lists via filtering/pagination, but it provides no explicit when-to-use guidance or alternatives. Since no exclusions are stated, the context is clear but minimal.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reject_triage_recommendationB
Destructive

Finalize a local triage proposal as rejected and record feedback.

ParametersJSON Schema
NameRequiredDescriptionDefault
actorYes
feedbackYes
ticketIdYes
recommendationIdYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
auditEventYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true, so the safety profile is known. The description adds minimal context by mentioning 'record feedback', but it does not explain consequences like irreversibility or state changes beyond what annotations imply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence that is front-loaded with the action and outcome. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the destructive nature and 4 required parameters, the description is too sparse. It does not explain side effects, prerequisites, or how rejection fits into the workflow, even though an output schema exists.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description must compensate, but it only implicitly references 'feedback'. It does not clarify the meaning or relationships of recommendationId, ticketId, or actor, leaving the agent to infer from parameter names alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Finalize') and identifies the resource ('local triage proposal') and the outcome ('as rejected and record feedback'). It clearly distinguishes this from sibling tools like approve_triage_recommendation and submit_triage_recommendation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives (e.g., approve_triage_recommendation). It simply states what the tool does without context about the decision process or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_knowledgeA
Read-onlyIdempotent

Search local support knowledge and policy articles.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
queryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
articlesYes

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, informing the agent this is a safe, repeatable read operation. The description adds the 'local' qualifier, indicating the search scope, but does not reveal behavior like result ranking or pagination. Given annotations, this is acceptable and not contradictory, but the added value is minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, front-loaded with the action, and contains no filler. It efficiently conveys the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple search tool, the description, combined with robust annotations and an output schema, is adequate. It specifies the resource type and scope. It lacks any caveats about search behavior, but the simplicity of the tool and presence of output schema compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has two parameters (query and limit) with zero schema descriptions. The description does not explain these parameters, leaving the agent to infer that 'query' is the search term and 'limit' caps results. This is a significant gap since the description carries full burden for param semantics with 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: searching local support knowledge and policy articles. The verb 'search' and the resource 'knowledge and policy articles' are specific and distinct from sibling tools that deal with tickets, queues, or audit events.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when this tool is appropriate: when the agent needs to search knowledge articles. However, it does not explicitly state when to avoid it or mention alternatives, such as find_similar_tickets for ticket similarity, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

submit_triage_recommendationA

Store a local triage proposal without changing the ticket or external systems.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNo
teamYes
actorYes
slaRiskYes
assigneeNo
categoryYes
priorityYes
ticketIdYes
rationaleYes
confidenceYes
outageRiskYes
securityRiskYes
ticketStatusNo
sourceRevisionYes
missingInformationYes
duplicateCandidatesYes
knowledgeArticleIdsYes
draftCustomerResponseYes
recommendedNextActionYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
recommendationYes

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds valuable behavioral context beyond the annotations: it explicitly states that the tool does not change the ticket or external systems, and that the proposal is local. This provides more specific non-destructive behavior than the annotations' destructiveHint=false. It could further disclose persistence or visibility details, but the key context is present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that delivers the core purpose effectively without unnecessary words. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (19 parameters, 16 required) and the presence of an output schema, the description is too minimal. It does not explain how this tool fits into the triage workflow (e.g., relation to approve/reject), what the parameters mean, or under what circumstances it should be called. The description is insufficient for an agent to correctly select and invoke the tool in context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for parameter meaning. However, the description provides zero details about any of the 19 parameters. It mentions 'triage proposal' but does not explain what fields like category, priority, team, or rationale signify, leaving the agent to rely solely on the schema's property names and enums.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Store') and resource ('local triage proposal'). It also explicitly notes that it does not change the ticket or external systems, which distinguishes it from sibling tools like approve_triage_recommendation and reject_triage_recommendation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage through 'without changing the ticket or external systems', but it does not explicitly state when to use this tool versus alternatives, nor does it mention any when-not-to-use conditions. The context is clear but the guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 9 tool updatesv1.0.0
    • First observedapprove_triage_recommendation
    • First observedfind_similar_tickets
    • First observedget_audit_events
    • First observedget_queue_metrics
    • First observedget_ticket
    • First observedlist_tickets
    • First observedreject_triage_recommendation
    • First observedsearch_knowledge
    • First observedsubmit_triage_recommendation

TDQS

A3.7/5.0

Scored across 9 tools

Disambiguation5/5

Each tool targets a distinct resource or action: list/get tickets differ from search/find, and the triage workflow has clear submit/approve/reject stages. No two tools share the same purpose or even overlap ambiguously.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case (list_tickets, search_knowledge, submit_triage_recommendation). The verbs are varied but predictable, and there are no mixed naming conventions or cryptic abbreviations.

Tool Count5/5

9 tools is well within the ideal 3-15 range. Each tool covers a necessary part of the ticket triage workflow—querying, analysis, auditing, and recommendation lifecycle—without redundancy or bloat.

Completeness4/5

The core triage workflow is well-covered: ticket retrieval, duplicate detection, knowledge search, metrics, audit, and recommendation submit/approve/reject. A minor gap is the lack of a dedicated tool to retrieve or list existing recommendations, which could force agents to rely on audit events.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    C
    quality
    C
    maintenance
    Incident Triage MCP is a Model Context Protocol (MCP) server for incident triage. It provides safe, auditable tools for evidence retrieval, deterministic summaries, ticket workflows, and notifications.
    28
    Apache 2.0
  • F
    license
    Not graded
    quality
    D
    maintenance
    A governed MCP server for integrating AI agents with customer data, featuring role-based access control, field redaction, and human-in-the-loop approval for secure support operations.
    1
    -
  • A
    license
    A
    quality
    B
    maintenance
    A local-first MCP server for retrieving a small evidence set and recording reviewed conclusions, policy-gated and redacted without giving an agent general filesystem access.
    5
    MIT
  • F
    license
    Not graded
    quality
    B
    maintenance
    MCP server that provides IT support tools to search known incidents, check ticket status, and create ticket drafts, enabling an AI assistant to help users with IT support queries.
    -