Honest Reasoning Engine — MCP Connector

Free, self-hosted MCP server. Attach it to any MCP-compatible AI client (Claude Desktop, Claude Code, Cursor, etc.) to get three tools that push back against a specific, measured failure mode: AI usage that quietly increases your workload instead of reducing it (surface-plausible output that isn't actually verified, decisional over-trust in AI claims, and review attention spent on the wrong things).

Free for personal/non-commercial use, no redistribution — see LICENSE.md. You run this yourself, with your own OpenAI API key — nothing is sent to any server we operate.

Tools

• honest_reason — structures any analysis/output-review/task around 4 disciplines: ruthless simplification, calibrated confidence labels (with a hard-coded guardrail — a model can't self-report "strong" confidence on fewer than 3 pieces of evidence, the code overrides it), falsification search, and honest limits.

• check_alignment_drift — logs every honest_reason call's "honest limits" locally, and detects when the same gap keeps recurring (e.g., "idempotency key never verified" showing up across many PRs). Pure embedding-similarity clustering, no LLM judgment, deterministic. Verified scope: catches literal/near-literal recurring gaps. Does not currently generalize across differently-worded gaps that share only an abstract category — that would need an added classification layer (not built yet).

• review_scope_card — turns a list of changed files/symbols (with reference counts you compute yourself, e.g. via grep) into a deterministic attention budget: what's worth reading carefully vs. safe to skim, given limited review time.

• gate_next_step — applies the free-energy-gate propose-and-gate principle to agent workflow steps: should you take one more step (extra tool call, retrieval, re-reasoning pass) or stop and answer at your current confidence? Deterministic, no LLM call (the gate itself must be cheap — it exists to decide whether to spend more compute). Takes your current confidence label (same vocabulary as honest_reason), what specific gap the next step would close, and its cost; rejects vague "let me just double-check" with no named target (scores it zero gain) and computes a step budget per context meant to prevent infinite loops. Also takes stakes (low/medium/high) as an axis independent of confidence — a "strong" confidence answer still gets pushed toward more verification if stakes are high, so the gate doesn't repeat the exact failure mode found in our own internal testing of honest_reason (a confident-sounding answer skipped a check it shouldn't have, on a high-stakes question). If the step budget runs out while stakes are high, it recommends escalating to a human instead of just answering. Caveat: the gain/cost/stakes scoring is a fixed table (see source), not something calibrated against real outcomes yet — treat the accept/reject boundary as a reasonable starting default, not a proven-optimal threshold. Second caveat (found via adversarial testing, 2026-07-15): the "is this gap specific enough" check is a cheap deterministic heuristic (minimum length + character diversity + reject simple repetition), not real semantic understanding — by design, since adding an LLM call here would defeat the point of a gate meant to decide whether to spend more compute. It correctly rejects naive padding (repeated characters/phrases) but a sufficiently varied wall of still-vague text, or a sufficiently character-diverse but meaningless string (e.g. asdfghjkl123), could in principle slip past it — an independent 4-lens expert review (decision theory / software engineering / agent systems / adversarial security, 2026-07-17) confirmed this by actually running such inputs through the function. Treat it as a filter for obvious gaming, not a guarantee; the scorecard in test/gate_scorecard.mjs reports its pass rate on cases other than these known-failing gibberish inputs, so don't read that number as overall robustness against all adversarial phrasing. Third caveat (same 2026-07-17 review, more serious): stepsSoFarInContext is not tracked server-side — gate_next_step is a stateless pure function, so the "step budget" only works if the calling agent honestly accumulates and passes this number itself. A non-cooperative or buggy caller can bypass it entirely by always passing

  1. acceptDelta and maxSteps are now clamped to fixed ranges ([-3,5] / [1,10]) server-side so a caller can no longer pass an extreme value (e.g. acceptDelta: -100) to force every call to "수락" regardless of the gain/cost math — that exact bypass was found and verified during the same review, and is fixed as of this commit. But the deeper point stands: every input to this gate (confidence, cost, stakes, step count) is self-reported by the same agent being gated, so it works best as a structured self-check for a cooperative agent, not as a security boundary against an adversarial or malfunctioning one.

Related MCP server: UltraThink

Setup

npm install

Set your OpenAI key (required — this connects to your OpenAI account, you pay for your own usage):

export OPENAI_API_KEY=sk-...        # macOS/Linux
$env:OPENAI_API_KEY = "sk-..."      # Windows PowerShell

Optional env vars:

• OPENAI_CHAT_MODEL (default gpt-4o-mini — cheaper; set to gpt-4o for higher quality)

• EMBED_MODEL (default text-embedding-3-small)

• HONEST_REASONING_DATA_DIR (default ~/.honest-reasoning — where check_alignment_drift's local signal log lives)

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "honest-reasoning": {
      "command": "node",
      "args": ["/absolute/path/to/mcp-server/index.js"],
      "env": { "OPENAI_API_KEY": "sk-..." }
    }
  }
}

Claude Code / Cursor (or any client that reads .mcp.json)

{
  "mcpServers": {
    "honest-reasoning": {
      "command": "node",
      "args": ["/absolute/path/to/mcp-server/index.js"],
      "env": { "OPENAI_API_KEY": "sk-..." }
    }
  }
}

Verify it works

node test/smoke.mjs /tmp/honest-reasoning-test-data

Runs review_scope_card and check_alignment_drift without needing an API key, plus confirms honest_reason gives a clear setup error if the key is missing. With OPENAI_API_KEY set, it also exercises the live LLM call.

Tools (continued)

• audit_discovery_claim — audits a claimed "AI/data discovered this pattern" against 5 standard statistical-methodology red flags: multiple comparisons, holdout/out-of-sample validation, effect-size-vs-complexity, alternative explanations, and reproducibility. Different from academic research-integrity tools (Statcheck, GRIM) which catch reporting errors — this catches internally-consistent but overfit/cherry-picked methodology instead. Hard guardrail: if holdout validation is marked insufficient, or if more than 5 candidates were considered without disclosed multiple-comparison correction, the overall verdict is mechanically capped below "reliable" regardless of what the model concludes. Honest limit: the guardrail-enforcement logic (audit.js) is unit-tested without needing an API key (test/audit_guardrail_test.mjs) — but the underlying judgment quality (does the model actually tell a real, holdout-validated finding apart from a fabricated one) has not been tested end-to-end yet, since no OPENAI_API_KEY was available in the environment this was built in. Treat it as an unverified first version until someone runs it live against a few known-good and known-bad cases.

When this actually helps (and when it doesn't)

Tested internally across personal-decision, business, and AI-output-review scenarios against a plain model call with no discipline applied. It's a clear improvement when the task is verifying a claim (yours or the AI's) or facing a decision with unstated assumptions — it surfaces what wasn't checked and ties confidence to actual evidence count instead of vibes.

It is not universally better. In one internal test (a personal-finance question involving a clear regulatory violation), a plain answer stated the hard "no — this is illegal, don't send it" directly. This tool's calibration instinct softened that into hedged confidence language and under-emphasized the violation. Lesson we're keeping visible rather than hiding: this is a lens for situations that need falsification-checking and calibration, not a replacement for a clear rule-based "stop" when one obviously applies.

What this is not

Not a fact-checker, not a source of truth. It structures how an AI reasons about uncertainty — confidence labels, falsification conditions, disclosed limits — it does not verify facts against the world. Don't use it as the sole basis for financial, medical, legal, or safety-critical decisions.