# Honest Reasoning Engine — MCP Connector

Free, self-hosted MCP server. Attach it to any MCP-compatible AI client
(Claude Desktop, Claude Code, Cursor, etc.) to get three tools that push
back against a specific, measured failure mode: **AI usage that quietly
increases your workload instead of reducing it** (surface-plausible output
that isn't actually verified, decisional over-trust in AI claims, and
review attention spent on the wrong things).

Free for personal/non-commercial use, no redistribution — see
[`LICENSE.md`](LICENSE.md). You run this yourself, with your own
OpenAI API key — nothing is sent to any server we operate.

## Tools

- **`honest_reason`** — structures any analysis/output-review/task around
  4 disciplines: ruthless simplification, calibrated confidence labels
  (with a hard-coded guardrail — a model can't self-report "strong"
  confidence on fewer than 3 pieces of evidence, the code overrides it),
  falsification search, and honest limits.
- **`check_alignment_drift`** — logs every `honest_reason` call's "honest
  limits" locally, and detects when the *same* gap keeps recurring
  (e.g., "idempotency key never verified" showing up across many PRs).
  Pure embedding-similarity clustering, no LLM judgment, deterministic.
  **Verified scope**: catches literal/near-literal recurring gaps. Does
  **not** currently generalize across differently-worded gaps that share
  only an abstract category — that would need an added
  classification layer (not built yet).
- **`review_scope_card`** — turns a list of changed files/symbols (with
  reference counts you compute yourself, e.g. via `grep`) into a
  deterministic attention budget: what's worth reading carefully vs. safe
  to skim, given limited review time.
- **`gate_next_step`** — applies the [free-energy-gate](https://github.com/NIWS-shindongju/free-energy-gate)
  propose-and-gate principle to agent workflow steps: should you take one
  more step (extra tool call, retrieval, re-reasoning pass) or stop and
  answer at your current confidence? Deterministic, no LLM call (the gate
  itself must be cheap — it exists to decide whether to spend *more*
  compute). Takes your current confidence label (same vocabulary as
  `honest_reason`), what specific gap the next step would close, and its
  cost; rejects vague "let me just double-check" with no named target
  (scores it zero gain) and computes a step budget per context meant to
  prevent infinite loops. Also takes `stakes` (low/medium/high) as an
  axis independent of confidence — a "strong" confidence answer still
  gets pushed toward more verification if stakes are high, so the gate
  doesn't repeat the exact failure mode found in our own internal testing
  of `honest_reason` (a confident-sounding answer skipped a check it
  shouldn't have, on a high-stakes question). If the step budget runs out
  while stakes are high, it recommends escalating to a human instead of
  just answering. **Caveat**: the gain/cost/stakes scoring is a fixed
  table (see source), not something calibrated against real outcomes yet
  — treat the accept/reject boundary as a reasonable starting default,
  not a proven-optimal threshold. **Second caveat (found via adversarial
  testing, 2026-07-15)**: the "is this gap specific enough" check is a
  cheap deterministic heuristic (minimum length + character diversity +
  reject simple repetition), not real semantic understanding — by design,
  since adding an LLM call here would defeat the point of a gate meant to
  decide whether to spend *more* compute. It correctly rejects naive
  padding (repeated characters/phrases) but a sufficiently varied wall of
  still-vague text, or a sufficiently character-diverse but meaningless
  string (e.g. `asdfghjkl123`), could in principle slip past it — an
  independent 4-lens expert review (decision theory / software engineering
  / agent systems / adversarial security, 2026-07-17) confirmed this by
  actually running such inputs through the function. Treat it as a filter
  for obvious gaming, not a guarantee; the scorecard in `test/gate_scorecard.mjs`
  reports its pass rate on cases *other than* these known-failing gibberish
  inputs, so don't read that number as overall robustness against all
  adversarial phrasing. **Third caveat (same 2026-07-17 review, more
  serious)**: `stepsSoFarInContext` is not tracked server-side — `gate_next_step`
  is a stateless pure function, so the "step budget" only works if the
  calling agent honestly accumulates and passes this number itself. A
  non-cooperative or buggy caller can bypass it entirely by always passing
  0. `acceptDelta` and `maxSteps` are now clamped to fixed ranges
  (`[-3,5]` / `[1,10]`) server-side so a caller can no longer pass an
  extreme value (e.g. `acceptDelta: -100`) to force every call to "수락"
  regardless of the gain/cost math — that exact bypass was found and
  verified during the same review, and is fixed as of this commit. But
  the deeper point stands: every input to this gate (confidence, cost,
  stakes, step count) is self-reported by the same agent being gated, so
  it works best as a structured self-check for a cooperative agent, not
  as a security boundary against an adversarial or malfunctioning one.

## Setup

```bash
npm install
```

Set your OpenAI key (required — this connects to *your* OpenAI account,
you pay for your own usage):

```bash
export OPENAI_API_KEY=sk-...        # macOS/Linux
$env:OPENAI_API_KEY = "sk-..."      # Windows PowerShell
```

Optional env vars:
- `OPENAI_CHAT_MODEL` (default `gpt-4o-mini` — cheaper; set to `gpt-4o` for higher quality)
- `EMBED_MODEL` (default `text-embedding-3-small`)
- `HONEST_REASONING_DATA_DIR` (default `~/.honest-reasoning` — where `check_alignment_drift`'s local signal log lives)

### Claude Desktop

Add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "honest-reasoning": {
      "command": "node",
      "args": ["/absolute/path/to/mcp-server/index.js"],
      "env": { "OPENAI_API_KEY": "sk-..." }
    }
  }
}
```

### Claude Code / Cursor (or any client that reads `.mcp.json`)

```json
{
  "mcpServers": {
    "honest-reasoning": {
      "command": "node",
      "args": ["/absolute/path/to/mcp-server/index.js"],
      "env": { "OPENAI_API_KEY": "sk-..." }
    }
  }
}
```

## Verify it works

```bash
node test/smoke.mjs /tmp/honest-reasoning-test-data
```

Runs `review_scope_card` and `check_alignment_drift` without needing an API
key, plus confirms `honest_reason` gives a clear setup error if the key is
missing. With `OPENAI_API_KEY` set, it also exercises the live LLM call.

## Tools (continued)

- **`audit_discovery_claim`** — audits a claimed "AI/data discovered this
  pattern" against 5 standard statistical-methodology red flags: multiple
  comparisons, holdout/out-of-sample validation, effect-size-vs-complexity,
  alternative explanations, and reproducibility. Different from academic
  research-integrity tools (Statcheck, GRIM) which catch *reporting errors*
  — this catches internally-consistent but *overfit/cherry-picked*
  methodology instead. Hard guardrail: if holdout validation is marked
  insufficient, or if more than 5 candidates were considered without
  disclosed multiple-comparison correction, the overall verdict is
  mechanically capped below "reliable" regardless of what the model
  concludes. **Honest limit**: the guardrail-enforcement logic (`audit.js`)
  is unit-tested without needing an API key (`test/audit_guardrail_test.mjs`)
  — but the underlying judgment quality (does the model actually tell a
  real, holdout-validated finding apart from a fabricated one) has **not**
  been tested end-to-end yet, since no OPENAI_API_KEY was available in the
  environment this was built in. Treat it as an unverified first version
  until someone runs it live against a few known-good and known-bad cases.

## When this actually helps (and when it doesn't)

Tested internally across personal-decision, business, and AI-output-review
scenarios against a plain model call with no discipline applied. It's a
clear improvement when the task is *verifying a claim* (yours or the AI's)
or facing a decision with unstated assumptions — it surfaces what wasn't
checked and ties confidence to actual evidence count instead of vibes.

It is **not** universally better. In one internal test (a personal-finance
question involving a clear regulatory violation), a plain answer stated
the hard "no — this is illegal, don't send it" directly. This tool's
calibration instinct softened that into hedged confidence language and
under-emphasized the violation. Lesson we're keeping visible rather than
hiding: this is a lens for situations that need falsification-checking and
calibration, not a replacement for a clear rule-based "stop" when one
obviously applies.

## What this is not

Not a fact-checker, not a source of truth. It structures *how* an AI
reasons about uncertainty — confidence labels, falsification conditions,
disclosed limits — it does not verify facts against the world. Don't use
it as the sole basis for financial, medical, legal, or safety-critical
decisions.