DisputeDesk MCP Server
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
DisputeDesk
Multi-agent card-dispute resolution with MCP tools, RAG, guardrails, observability and evals built in. LangGraph + FastAPI, runs locally, deploys to AWS.
Portfolio project. All data is synthetic. See PROJECT_BLUEPRINT.md for the architecture and milestones.
Status
See PROGRESS.md for milestone status and details.
Related MCP server: Agentic RAG MCP Server
Quickstart
Requires uv and Docker.
cp .env.example .env
docker compose up -d --wait # Postgres+pgvector, Langfuse (:3000), OTel collector (:4317/4318)
uv sync --all-extras --dev
uv run pytest -qLLM_PROVIDER=fake (the default in .env.example) needs no API key. Set LLM_PROVIDER=anthropic and
ANTHROPIC_API_KEY to use Claude, or LLM_PROVIDER=ollama for a local model.
Evals
uv run python evals/run_evals.py --thresholds evals/thresholds.yml runs 42 synthetic disputes (10
adversarial) through the real graph and exits non-zero if any threshold fails. Needs a real model and
the rulebook ingested (uv sync --all-extras --dev --group evals, uv run python -m disputedesk.rag.ingest).
A full run takes about 30 minutes.
Latest result (Haiku 5.5 as system and judge, 2026-11-20). The gate is red (3 of 9 thresholds fail): the system is not good enough yet, and the thresholds were fixed before the first run. Details, causes and what changed between runs are in PROGRESS.md.
Metric | Baseline | Now | Threshold |
Router accuracy | 1.000 | 1.000 | ≥ 0.90 |
Decision accuracy (strict) | 0.133 | 0.781 | ≥ 0.80 |
Human-approval routing accuracy | 1.000 | 1.000 | ≥ 1.00 |
Citation hit rate | 0.133 | 0.969 | ≥ 0.70 |
Citation faithfulness (DeepEval) | 0.952 | 0.969 | ≥ 0.85 |
Letter quality (LLM judge, 1-5) | 3.23 | 4.03 | ≥ 4.00 |
Guardrail catch rate | 0.333 | 0.333 | ≥ 0.80 |
Guardrail false-positive rate | 0.033 | 0.031 | ≤ 0.05 |
Injection pass rate | 0.400 | 0.900 | ≥ 1.00 |
Cases that crashed | 20 of 40 | 0 of 42 |
Red-team: uv run npx promptfoo@0.126.0 eval -c evals/promptfoo/redteam.yaml --no-cache (16 of 18 pass after M7.2, up from 4; not re-run since).
Development
uv run ruff check . && uv run ruff format .
uv run mypy src
uv run pytest tests/unit -q
uv run pre-commit install # optional: run the same checks on every commitArchitecture decisions
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Build and manage AI-native customer support agents from Claude or any MCP client.
MCP server for building and testing AI agents with multi-model experimentation and insights.
An MCP gateway for teams deploying AI tools at scale to employee-facing and headless agents.
AgentGuard — 20-tool AI safety MCP: policy preflight, risk scoring, audit logging, rate limits.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceA production-grade MCP server that exposes real-time banking data replicated via Oracle GoldenGate CDC as structured tools for AI agents, enabling read, score, and write operations on customer, account, transaction, and alert data.1-
- AlicenseNot gradedqualityDmaintenanceMCP server providing tools for entity extraction, query refinement, and relevance checking to build Agentic RAG applications.MIT
- FlicenseNot gradedqualityDmaintenanceMCP server that provides tools for retrieving transaction context and recording AI decisions or creating human reviews for payment risk exceptions. Enables LLM agents to handle exception transactions in a hybrid payment-decisioning workflow.-
- AlicenseNot gradedqualityDmaintenanceAn MCP-native agentic platform orchestrating planner/executor/critic agents over hybrid RAG with three-tier memory, budget enforcement, safety guardrails, and full observability. It exposes all capabilities as MCP tools, enabling natural-language control of document ingestion, retrieval-augmented generation, and multi-step AI workflows.MIT