mythos-reasoning-mcp
by sorryorc
README.md
# Mythos Reasoning MCP
[](https://www.npmjs.com/package/mythos-reasoning-mcp)
[](LICENSE)
[](#tools)
[](#what-is-new-in-05)
[](#)
> **A deterministic verification gate for Codex / Claude Code / any MCP client — independent of the model under check.**
> Mythos does not make the model smarter. It checks the model's output against evidence, contradictions, calibration, and provenance — because a model's self-assessment is exactly what should not be trusted blindly.
---
## TL;DR
```text
Before: model -> answer -> ship ❓ (self-graded, no independent check)
After: model -> answer -> Mythos gate -> ready | revise | gather_evidence | formalize | block
```
- **19 tools**, single MCP server, zero LLM and zero network by default.
- **Tier 0 (default)**: fully local heuristics — claim verification, contradiction detection, calibration (Brier + ECE), semantic entropy, provenance binding.
- **Tier 1 (opt-in)**: optional `mythos_judge` for fresh-context semantic verdicts. Disabled unless you set an API key.
- **Measurable**: `npm run eval:gate` evaluates a labeled dataset and writes precision / recall / F1 / claim-level recall to `reports/gate_eval.*`.
---
## Quick start
```bash
git clone https://github.com/sorryorc/mythos-mcp.git
cd mythos-mcp
npm install
npm run build
npm test
```
Wire it into Codex (or any MCP client) by pointing at the built entry:
```toml
[mcp_servers.mythos_reasoning]
command = "node"
args = ["<absolute path to>/mythos-mcp/dist/index.js"]
cwd = "<absolute path to>/mythos-mcp"
enabled = true
startup_timeout_sec = 20
tool_timeout_sec = 120
```
When your client exposes tools named like `mcp__mythos_reasoning__mythos_optimize`, the MCP is connected.
---
## What it actually does
Mythos Reasoning MCP turns the `Prelude → recurrent depth → Coda` idea into a practical workflow:
```text
plan -> recurrent depth (to convergence) -> verify + calibrate -> detect contradictions -> critique + probe -> gate -> verified coda
```
It runs **no LLM** and makes **no network calls** on the default path. It exposes auditable summaries — confidence scores, evidence coverage, calibration, contradiction reports, risk flags, and token-budget estimates — never private chain-of-thought.
> **Positioning:** "Mythos" here is a **deterministic verification layer**, not a specific model brand. Modern reasoning models already supply depth and self-validation natively; this layer does not replace that — it *independently* checks the model's output, because a model's own (potentially grader-aware) self-assessment is exactly what should not be trusted blindly.
## What is new in 0.5
0.6 builds on the v0.5 refocus with measurable gate evaluation, deterministic provenance binding, and claim routing. The public tool surface is now 19 tools: the quality pipeline, deeper verification helpers, provenance/route checks, the optional judge, and progress authenticity audit.
- **Refocused tool surface**: removed context tiering, tool orchestration, memory, scaffolding, agent coordination, protocol batching, security lattice, and observability tuning tools.
- **Verification-first shape**: retained budget, plan/pass/run, verify/critique/finalize, gate/optimize, compare, formalize, consistency, assumptions, probe, verify-chain, judge, provenance, route, and audit-progress.
- **Gate effectiveness is measurable**: `npm run eval:gate` evaluates `datasets/labeled.sample.json` and writes precision/recall/false-block/F1 plus claim-level recall to `reports/gate_eval.*`.
- **Provenance and routing**: `mythos_check_provenance` flags factual claims without verifiable pointers; `mythos_route` decides which claims stay on Tier 0, need Tier 1 judge, or need more evidence.
- **Docs and tests match the runtime**: unit tests no longer cover deleted runtime primitives, and compliance audit scans the remaining 3 source files that contain public tool descriptions or judge prompts.
## What was new in 0.4
0.4 retargets the layer for **Mythos 5 / Fable 5-class models**, whose native effort dial supplies the reasoning depth this MCP used to simulate. Value moves to what the model's own self-assessment cannot be trusted to do — independent verification, calibration, and gating.
- **Budget → effort mapping** (`mythos_budget`): each active mode maps onto the host model's test-time-compute effort dial — `lite → low`, `standard → high`, `deep → xhigh`, `max → max`; `off` skips the host-model call — with an `effort_rationale`. Depth comes from the model's effort; the MCP governs *when* that spend is worth it. The recurrent-pass loop is kept only as a budget/convergence governor (`stalled` ⇒ gather evidence, don't spend more effort).
- **Multi-agent merge gate** (`mythos_gate` merge mode): pass `candidates` (≥2 sub-agent outputs) and the gate runs cross-agent consistency + contradiction detection and gates the *merge* (`merge_ready` / `reconcile` / `block_merge`) — the deterministic check where async sub-agents (e.g. Git-shared worktrees) converge. A high-severity cross-agent contradiction forces `block_merge`.
- **Response provenance awareness**: `mythos_run`, `mythos_gate`, and `mythos_optimize` accept `response_source` (`fable5 | opus_fallback | unknown`) plus `stop_reason`. Opus fallback paths are reported in `provenance`; high-risk gates tighten the quality bar and require fallback-aware release notes.
- **Reasoning-extraction compliance audit**: `npm test` runs `dist/tests/compliance_audit.js`, which scans tool descriptions and prompt templates for unsafe requests to reveal, repeat, transcribe, or extract private reasoning.
- **Tier 1 judge tool**: `mythos_judge` exposes the `JudgeClient` adapter with disabled, mock, and Anthropic HTTP implementations. It supports multi-sample self-checks, semantic-entropy stability, optional cross-model second pass, and per-session cache keys. Without `ANTHROPIC_API_KEY`, it returns a disabled result and Tier 0 remains the default zero-network path.
- **Progress authenticity audit**: `mythos_audit_progress` checks a model's progress report against real tool-result logs and flags unsupported or unverifiable progress claims.
## What is new in 0.3
0.3 mines established techniques from the literature and folds them into the deterministic, no-LLM core:
- **Chain-of-Verification** (`mythos_verify_chain`): generate independent verification questions per claim / risk-tag / specific, to answer *before* finalizing (Dhuliawala et al., 2023).
- **Semantic entropy** in `mythos_consistency`: cluster candidate answers by meaning and report normalized entropy + cluster stats — a stronger confabulation signal than lexical overlap (Farquhar et al., Nature 2024).
- **Atomic-fact decomposition** (FActScore, Min et al., 2023): split sentences into atomic claims so one false sub-clause is not hidden inside a true sentence; surfaced as `factual_precision`.
- **Proper calibration**: the calibration report now adds a Brier score and Expected Calibration Error (ECE) over claims.
- **Configurable adaptive exit** (Huginn, Geiping et al., 2025): `mythos_run` accepts `exit_criterion` (`latent_diff` default, or `ema`) and `exit_threshold`, mapping their latent/KL exit idea onto the belief iteration.
- **Front-loaded verification**: `mythos_optimize` attaches the CoVe questions (`verification_questions`) and folds the top ones into `codex_instruction`, so the guardrail shapes the answer *before* it is written — not only after.
## What is new in 0.2
Version 0.1 had a placeholder "recurrent" loop that just added a fixed `pass * 0.04` to a confidence number every iteration — monotonic by construction and carrying no information. 0.2 replaces the core and adds three analysis tools:
- **True recurrent depth.** The reasoning state relaxes toward a fixed point: `belief_t = belief_{t-1} + lr * (target(state) - belief_{t-1})`, where `target` is derived from evidence strength, open unknowns, and detected contradictions. Iteration halts on convergence, oscillation, or stall.
- **Adaptive compute.** Harder inputs (weak evidence, more unknowns/contradictions) get a lower learning rate and a larger pass budget, so the engine spends more passes where it matters and fewer on easy inputs.
- **Contradiction detection.** Negation, modal (`always` vs `sometimes`), numeric/version, and claim-vs-evidence mismatches.
- **Calibration.** Each claim's *stated* confidence is compared against its *evidence-derived* confidence; systematic overconfidence is penalized.
- **New tools:** `mythos_consistency` (self-consistency / ensemble), `mythos_assumptions` (assumption mining), `mythos_probe` (edge-case / counterfactual probing).
## Codex Status
This project is installed at:
```text
<your local path>/mythos-mcp
```
Codex config:
```toml
[mcp_servers.mythos_reasoning]
command = "node"
args = ["<your local path>/mythos-mcp/dist/index.js"]
cwd = "<your local path>/mythos-mcp"
enabled = true
startup_timeout_sec = 20
tool_timeout_sec = 120
```
When Codex exposes tools named like `mcp__mythos_reasoning__mythos_optimize`, the MCP is connected.
## Tools
The v0.6 public surface is 19 tools.
Quality pipeline:
- `mythos_budget`: estimate overhead, recommend `off`/`lite`/`standard`/`deep`/`max`, and map active modes to the host model effort dial (`low`/`medium`/`high`/`xhigh`/`max`).
- `mythos_plan`: classify task mode, risk level, risk factors, and evidence needs.
- `mythos_pass`: run one recurrent relaxation step (carries state across turns).
- `mythos_verify`: score claims by evidence support; report per-claim status, risk tags, and calibration.
- `mythos_critique`: find unsupported claims, contradictions, overconfidence, miscalibration, missing verification, and edge-case gaps.
- `mythos_finalize`: produce a concise answer with evidence summary, contradictions, and uncertainty notes.
- `mythos_run`: run the full workflow (plan -> depth -> verify -> contradictions -> critique -> finalize -> score).
- `mythos_gate`: decide `ready` / `revise` / `gather_evidence` / `formalize` / `block`. **Merge mode** (`candidates` >= 2): gate a multi-agent merge with `merge_ready` / `reconcile` / `block_merge`.
- `mythos_formalize`: turn an unresolved problem into variables, constraints, an objective, and next actions ranked by expected value.
- `mythos_optimize`: single-entry orchestration for Codex (budget, gate, formalize, assumptions, open edge cases, evidence requests, output contract).
- `mythos_compare`: compare baseline output against a Mythos-checked answer.
Analysis tools (deeper thinking):
- `mythos_consistency`: self-consistency / ensemble over multiple candidate answers — agreement score, consensus claims, and cross-candidate contradictions.
- `mythos_assumptions`: mine the hidden assumptions a draft depends on and rank them by `risk_if_false / cost`.
- `mythos_probe`: generate the edge cases / counterfactuals an answer must survive, flagging which are already addressed.
- `mythos_verify_chain`: Chain-of-Verification — independent verification questions for a draft's claims, prioritized by weakness and risk.
- `mythos_judge`: optional Tier 1 fresh-context semantic judge for claim/evidence entailment (`supported | contradicted | unsupported`). Defaults to disabled without an API key; use `provider: "mock"` for local deterministic testing. Supports `sample_count`, `entropy_threshold`, `cross_model`, `secondary_model`, and `use_cache`.
- `mythos_check_provenance`: deterministically checks whether claims are bound to verifiable pointers (`file_line`, `test_id`, `url`, or `quote`) and counts unbound specific claims.
- `mythos_route`: routes claims to `tier0_only`, `tier1_judge`, or `gather_evidence` and estimates judge cost before spending it.
- `mythos_audit_progress`: compare progress statements to real tool results and return grounded / false / unverifiable labels plus floating claims.
Resources: `mythos://protocol` (the workflow) and `mythos://depth` (the recurrent-depth math).
## Recurrent depth and adaptive compute
```text
belief_t = belief_{t-1} + lr * (target(state) - belief_{t-1})
target(state) = base(evidence_strength) - unknown_penalty - contradiction_load
lr = 0.85 - 0.45 * difficulty # harder inputs creep, using more passes
```
Halting (`halted_reason`):
- `converged` — `|delta| < epsilon` with belief high enough.
- `stalled` — belief stopped moving but unknowns/contradictions remain (needs new evidence, not more passes).
- `oscillation` — the target keeps flipping across passes (conflicting evidence).
- `max_passes` — budget exhausted; `recommended_additional_passes` estimates what remains.
The depth result is auditable: each pass reports `belief`, `target`, `delta`, contradiction count, and open checks.
## Verification, contradictions, and calibration
`mythos_verify` assigns each claim an evidence-derived confidence from a single documented weight table (`test_result` > `file_line` > `docs` > `web` > `user_input` > `inference` > `unknown`), adjusted for a cited source, absolute language, and bare specifics. Status bands:
- `supported`: strong, sourced, typed evidence.
- `needs_more_evidence`: plausible but not proven (e.g. a cited inference).
- `unsupported`: no usable evidence.
Calibration compares stated vs evidence-derived confidence and reports `overconfident_claims` and a `calibration_score`. Note that negated absolutes (`cannot guarantee`, `not always`) are treated as hedging, not overconfidence.
## Budget Modes
| Mode | Model effort | Typical use | Passes | Estimated overhead |
| --- | --- | --- | ---: | ---: |
| `off` | skip call | Casual chat, creative drafting, low-risk preference questions | 0 | 0 |
| `lite` | low | Short factual answers, quick final check | 1 | about 180 tokens + 15% of checked text |
| `standard` | high | Normal coding, debugging, current information, design tradeoffs | 2 | about 420 tokens + 35% of checked text |
| `deep` | xhigh | Security, legal, finance, auth, destructive actions, costly decisions | 4 | about 900 tokens + 80% of checked text |
| `max` | max | Highest-capability sensitive tasks where the extra cost/latency is justified | 5 | about 1300 tokens + 110% of checked text |
For best value, start with `mythos_budget`. Use `mythos_run` / `mythos_optimize` when the recommendation is `standard` or `deep`, or when the answer contains important factual claims.
`mythos_run`, `mythos_gate`, and `mythos_optimize` can also receive:
```json
{
"response_source": "fable5 | opus_fallback | unknown",
"stop_reason": "optional provider stop/refusal reason"
}
```
Use `opus_fallback` when the host fell back from the requested Fable 5 / Mythos 5-class path. The gate does not blindly reject fallback output, but it reports a downgraded provenance posture and is stricter on high-risk tasks.
## Optional Tier 1 Judge
The judge layer is exposed through `mythos_judge` and routed by `mythos_route`. The default `mythos_gate` path remains deterministic and zero-network; `mythos_optimize` reports which claims justify Tier 1 and the estimated spend.
Environment contract:
```text
ANTHROPIC_API_KEY enables the Anthropic judge client
MYTHOS_JUDGE_PROVIDER=mock uses the local deterministic mock client
MYTHOS_JUDGE_PROVIDER=disabled forces Tier 1 off
MYTHOS_JUDGE_MODEL defaults to claude-fable-5
MYTHOS_JUDGE_EFFORT low | medium | high | xhigh | max, defaults to high
MYTHOS_JUDGE_MAX_OUTPUT_TOKENS defaults to 1200
MYTHOS_JUDGE_TIMEOUT_MS defaults to 30000
MYTHOS_JUDGE_ENDPOINT defaults to https://api.anthropic.com/v1/messages
```
The Anthropic request uses the Messages API with `model`, `max_tokens`, `messages`, and `output_config.effort`; it sends the required `anthropic-version: 2023-06-01` header. The judge prompt asks only for structured JSON verdicts and explicitly avoids private reasoning.
Example local call:
```json
{
"task": "Judge whether claims are supported.",
"provider": "mock",
"sample_count": 2,
"use_cache": true,
"claims": [
{
"claim": "The unit tests pass.",
"evidence_type": "test_result",
"source": "npm test"
}
]
}
```
For progress reports, pass either `statements` or `progress_report` with compact `tool_results`:
```json
{
"statements": ["I ran npm test.", "I deployed to production."],
"tool_results": [
{ "id": "t1", "tool": "shell", "output": "npm test passed all checks" }
]
}
```
## Recommended Codex workflow
```text
1. Use normal local tools first: read files, run tests, inspect docs.
2. Call mythos_optimize with task + draft + compact claims/evidence.
3. ready -> answer under output_contract.
4. gather_evidence-> collect the requested evidence first.
5. formalize -> follow the ranked next step.
6. revise -> rewrite under the output rules.
7. block -> do not answer; resolve the contradiction first.
```
Use the lower-level tools (`mythos_verify`, `mythos_critique`, `mythos_consistency`, `mythos_assumptions`, `mythos_probe`, `mythos_pass`) when debugging the pipeline itself or when you want one specific check.
## Maximum benefit with controlled cost
```text
skip: casual/creative/no factual stakes
lite: short answers with 1-3 claims
standard: coding, debugging, docs, current info, architecture decisions
deep: security, credentials, payments, destructive operations, legal/medical/financial risk
max: highest-capability sensitive tasks after the deterministic gate justifies the spend
```
Send compact drafts, claims, and evidence summaries — not entire files, logs, or transcripts. The MCP cannot retrieve sources or run tests; it makes evidence gaps, contradictions, and overconfidence visible before the final answer.
## Replay evaluation
Record real Codex tasks after an answer attempt:
```powershell
npm run replay:record -- replay/sample_record.json replay/replay.jsonl
npm run replay:analyze -- replay/replay.jsonl
npm run replay:report -- replay/replay.jsonl reports/replay.md
npm run replay:reset -- replay/replay.jsonl
```
Records include task, draft, claims, evidence, outcome, and notes. Use outcomes consistently: `accepted | reworked | failed | blocked | unknown`. The strongest tuning signal is `ready` decisions that later become `failed` or `reworked` — that means the gate is too permissive for that task class. (`reset` truncates the log even on filesystems that block file deletion.)
## References
The deterministic heuristics are inspired by — not implementations of — these works:
- Geiping et al., "Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach" (arXiv:2502.05171, 2025).
- Dhuliawala et al., "Chain-of-Verification Reduces Hallucination in Large Language Models" (arXiv:2309.11495, 2023).
- Farquhar, Kossen, Kuhn & Gal, "Detecting hallucinations in large language models using semantic entropy", Nature 630 (2024).
- Min et al., "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation" (EMNLP 2023).
- Wang et al., "Self-Consistency Improves Chain-of-Thought Reasoning in Language Models" (arXiv:2203.11171, 2022).
## Test
```bash
npm test
```
`npm test` builds, runs the MCP smoke test, the v0.2 unit suite (recurrent convergence, contradiction detection, calibration, gate decisions, and the new tools), the comparison and dataset evaluations, and a replay round-trip. Reports are written to:
```text
reports/comparison.md
reports/real_tasks.sample.eval.md
reports/gate_eval.md
reports/test_replay.md
```
This server cannot be deployed
Maintenance
ActivityInactive
ResponsivenessNo issues