# Claude Text Lab

[![tests](https://github.com/cuibitlabs/claude-watermark-remover/actions/workflows/tests.yml/badge.svg)](https://github.com/cuibitlabs/claude-watermark-remover/actions/workflows/tests.yml)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![MCP server](https://img.shields.io/badge/MCP-stdio%20server-orange.svg)](docs/mcp.md)
[![provenance verdict: unverified](https://img.shields.io/badge/provenance%20verdict-unverified-lightgrey.svg)](#read-this-first)

**Fact-preserving regeneration of Claude-generated text, with auditable local quality gates and an honest `unverified` provenance verdict.** No detector is claimed, and none exists publicly.

Claude Text Lab 0.1.1 is a local-first Python research toolkit for substantial, fact-preserving regeneration of Claude-generated text. It separates the source into a reviewed fact ledger, writes new prose from that ledger with a non-Claude model, checks factual invariants and wording overlap, and records the evidence in JSON.

This project is Claude text only. It does not process images, video, audio, PDFs, DOCX files, C2PA metadata, Gemini, SynthID, or visible logos.

## Read this first

As of the August 13, 2026 research cutoff, Anthropic says supported Claude models embed an imperceptible watermark directly into generated text. Anthropic has not published the algorithm, detector keys, thresholds, error rates, minimum reliable length, or a public detector API. No third-party program can currently prove that a Claude text watermark was removed.

For that reason, every Claude Text Lab result keeps this verdict:

```text
provenance_verdict: unverified
```

A passing local quality gate means the candidate preserved the checked facts, avoided suspicious Unicode, stayed within the configured length range, and did not reuse too much source wording. It does not mean an Anthropic detector returned a negative result.

Anthropic states that heavy editing, paraphrasing, translation, mixing with other writing, and very short passages can leave no detectable mark. Anthropic also warns that a negative detection result would not prove that Claude was never involved. See [Anthropic's current marking documentation](https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content).

## Choose the right command

| Goal | Command | Uses a model | What it proves |
|---|---|---:|---|
| List suspicious Unicode and protected values | `inspect` | No | What the local scanner found |
| Normalize or remove suspicious Unicode | `sanitize` | No | Which deterministic text changes were made |
| Compare an existing rewrite with its source | `audit` | No | Whether configured local quality gates pass |
| Create independent prose from a fact ledger | `rewrite` | Yes | Transformation and review evidence only |
| Evaluate many source and candidate pairs | `benchmark` | No | Aggregate local metrics and supplied detector observations |
| Expose the same operations to an MCP host | `claude-text-lab-mcp` | Sometimes | The same evidence as the corresponding functions |

If you only want ordinary Unicode cleanup, use `sanitize`. If your concern is Claude's announced model-level text mark, Unicode cleanup alone is not enough. Use `rewrite`, then inspect the ledger, candidate, and report manually.

## How the rewrite works

```mermaid
flowchart TD
    A["Authorized source text"] --> B["Conservative Unicode normalization"]
    B --> C["Non-Claude fact extraction"]
    C --> D["Ledger review"]
    D --> E["Fresh prose from ledger only"]
    E --> F["Exact-value and overlap gates"]
    F --> G["Semantic verification"]
    G -->|failed review| E
    G --> H["Candidate, ledger, and JSON report"]
```

The writing stage never receives the original prose. It sees atomic claims, exact values, uncertainties, and format requirements from the reviewed ledger. The source is still sent to the extraction and ledger-review stages, and the selected model endpoint therefore needs permission to process it.

With default settings, a successful first candidate usually takes four model calls:

1. Extract a fact ledger from the source.
2. Review the ledger against the source.
3. Compose fresh prose from the ledger only.
4. Verify the candidate against the ledger.

If the first candidate fails, a second candidate is generated with feedback. `--fast` uses one extraction call and one writing call, but skips ledger review and semantic verification. Use fast mode for experiments, not important content.

## Requirements

- Python 3.10 or newer.
- UTF-8 plain text input.
- A non-Claude model behind an OpenAI-compatible chat-completions endpoint for `rewrite`.
- Enough model quality and memory for accurate fact extraction and verification.

The core CLI, inspection, sanitation, audit, and benchmark functions use only the Python standard library. MCP is an optional dependency.

## Installation

### Linux and macOS

From the extracted release directory or a Git checkout:

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .
claude-text-lab --version
```

Expected version:

```text
claude-text-lab 0.1.1
```

For editable development:

```bash
python -m pip install -e '.[dev,mcp]'
```

### Windows PowerShell

```powershell
py -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install .
claude-text-lab --version
```

If PowerShell blocks virtual-environment activation, either use the full executable path, `.venv\Scripts\python.exe`, or review your local execution policy before changing it.

### Install the built wheel

The release bundle contains a wheel in `dist/`:

```bash
python -m pip install dist/claude_text_lab-0.1.1-py3-none-any.whl
```

The base wheel does not install MCP. Install from the project with the `mcp` extra when MCP is needed:

```bash
python -m pip install '.[mcp]'
```

## Set up a local non-Claude model

Any correctly implemented OpenAI-compatible `/chat/completions` endpoint can work. The examples use Ollama because it can run locally. The model name is an example, not an endorsement or a claim that it is suitable for every language or domain.

1. Install [Ollama](https://ollama.com/download).
2. Download a non-Claude model:

   ```bash
   ollama pull qwen3:8b
   ```

3. Confirm the model is present:

   ```bash
   ollama list
   ```

4. Confirm the OpenAI-compatible endpoint responds:

   ```bash
   curl http://127.0.0.1:11434/v1/models
   ```

Ollama documents its OpenAI-compatible endpoint at [docs.ollama.com](https://docs.ollama.com/api/openai-compatibility), and the `qwen3:8b` model is listed in the [Ollama model library](https://ollama.com/library/qwen3:8b).

If a different local server is used, replace both `--endpoint` and `--model`. The endpoint may be a base URL such as `http://127.0.0.1:1234/v1` or a complete URL ending in `/chat/completions`.

## Complete first rewrite tutorial

### 1. Create an input file

Copy only text you own or are authorized to transform into `source.txt`. A sample is included at `examples/source.txt`.

```text
The Atlas documentation release is scheduled for August 13, 2026. The public status page is https://status.example.com, and checks run every 30 minutes. The schedule remains provisional until final review.
```

### 2. Inspect the source

```bash
claude-text-lab inspect source.txt --json
```

Review `suspicious_unicode`, `exotic_spaces`, and `protected_spans`. An empty suspicious-Unicode list does not say anything about a statistical text watermark.

### 3. Run the full rewrite

```bash
claude-text-lab rewrite source.txt \
  -o regenerated.txt \
  --ledger facts.json \
  --report audit.json \
  --endpoint http://127.0.0.1:11434/v1 \
  --model qwen3:8b \
  --candidates 3 \
  --style "concise technical documentation" \
  --fail-on-review
```

The tool refuses model identifiers containing `claude` or `anthropic`. Rewriting with a supported Claude model could add a new Claude mark.

The command creates:

| File | Purpose |
|---|---|
| `regenerated.txt` | Best candidate selected by deterministic ranking |
| `facts.json` | Reviewed claims, exact values, uncertainties, and format requirements |
| `audit.json` | Configuration, hashes, all candidate results, local gates, semantic review, and `unverified` provenance verdict |

When output is written to a file, the JSON report is also printed to standard output. Use `--report` to retain a stable copy.

### 4. Review the ledger before trusting the candidate

Open `facts.json` and check that:

- Every material claim is present.
- Attribution, negation, uncertainty, and causal direction are correct.
- Names, dates, numbers, URLs, commands, quotations, and identifiers are exact.
- No instruction embedded in the source became a fact.

Then compare `regenerated.txt` with the source for tone and domain accuracy. The model verifier is useful, but it is not a substitute for a subject-matter reviewer.

### 5. Read the report correctly

The most important fields are:

| Field | Interpretation |
|---|---|
| `accepted` | The chosen candidate passed the configured local quality and semantic checks. |
| `selected_attempt` | Candidate number selected by deterministic ranking. |
| `ledger_reviewed` | Whether the second model-assisted ledger pass ran. |
| `missing_protected_tokens` | Exact values that disappeared. |
| `added_protected_tokens` | New protected values introduced by the candidate. |
| `source_ngram_reuse` | Fraction of unique source word n-grams found in the candidate. |
| `longest_shared_word_run` | Longest consecutive word sequence shared with the source. |
| `semantic_verification.verdict` | `pass`, `review`, `fail`, or `not_run`. |
| `provenance_verdict` | Always `unverified` without an authoritative Claude detector. |
| `source_sha256`, `output_sha256`, `ledger_sha256` | Reproducibility hashes for the run artifacts. |

### 6. Understand exit code 3

With `--fail-on-review`, exit code `3` means none of the generated candidates passed every configured gate. The best candidate and requested reports are still written so they can be diagnosed. Do not publish that candidate without resolving the reported problems.

## Command tutorials

### Inspect

```bash
claude-text-lab inspect source.txt --json
```

`inspect` reports document counts, protected values, suspicious Unicode, and exotic spaces. It never makes a detector claim. Use `-` for standard input:

```bash
printf 'Example text' | claude-text-lab inspect - --json
```

### Sanitize Unicode

Conservative sanitation:

```bash
claude-text-lab sanitize source.txt \
  -o normalized.txt \
  --report unicode-report.json
```

The default mode applies NFC normalization, turns exotic spaces into ordinary spaces, removes common zero-width, bidi, tag, noncharacter, and unsafe control code points, and preserves context-sensitive joiners and variation selectors.

Aggressive mode also removes all format controls and variation selectors:

```bash
claude-text-lab sanitize source.txt -o normalized.txt --aggressive --force
```

Aggressive mode can damage scripts that rely on joiners or emoji variation selectors. Inspect and review the result. Neither mode removes a token-choice watermark.

Use `--no-normalize-spaces` to preserve exotic spaces. Existing files are protected from replacement unless `--force` is supplied.

### Audit an existing rewrite

```bash
claude-text-lab audit source.txt regenerated.txt --json
```

Add exact values that must appear:

```bash
claude-text-lab audit source.txt regenerated.txt \
  --require 'August 13, 2026' \
  --require 'https://status.example.com' \
  --json
```

`audit` reports a failed gate in JSON but still exits successfully unless an input or configuration error occurs. Read `quality_gate.passed`; do not use the process exit code as the audit verdict.

Default gate settings are:

| Setting | Default | Meaning |
|---|---:|---|
| `--ngram-size` | `5` | Word sequence length used for reuse measurement |
| `--max-ngram-reuse` | `0.08` | Maximum allowed fraction of source n-grams reused |
| `--max-shared-run` | `12` | Maximum consecutive source words allowed |
| `--min-length-ratio` | `0.65` | Minimum candidate words divided by source words |
| `--max-length-ratio` | `1.5` | Maximum candidate words divided by source words |

These defaults are engineering heuristics, not Anthropic detector thresholds.

### Rewrite options

```bash
claude-text-lab rewrite --help
```

| Option | Purpose |
|---|---|
| `-o`, `--output` | Required candidate file, or `-` for standard output |
| `--ledger` | Save the reviewed fact ledger as JSON |
| `--report` | Save the complete run report as JSON |
| `--endpoint` | Writer model endpoint, default `http://127.0.0.1:11434/v1` |
| `--model` | Writer model identifier, default `qwen3:8b` |
| `--api-key` | Writer endpoint key, default `local` |
| `--verifier-endpoint` | Optional separate verifier endpoint |
| `--verifier-model` | Optional separate verifier model |
| `--verifier-api-key` | Optional separate verifier key |
| `--allow-remote` | Permit non-loopback HTTPS endpoints |
| `--style` | Requested style for the independent composition |
| `--language` | Requested output language |
| `--candidates`, `--passes` | Maximum candidate attempts, from 1 through 20 |
| `--no-ledger-review` | Skip the second ledger check |
| `--no-semantic-check` | Skip model-assisted candidate verification |
| `--allow-added-protected` | Permit new protected numbers, dates, URLs, and similar values |
| `--fast` | One candidate, no ledger review, no semantic verification |
| `--fail-on-review` | Exit with code 3 when no candidate passes |
| `--force` | Replace existing requested outputs |

Use a separate verifier when factual accuracy matters and a second capable local model is available:

```bash
claude-text-lab rewrite source.txt -o regenerated.txt \
  --endpoint http://127.0.0.1:11434/v1 \
  --model qwen3:8b \
  --verifier-endpoint http://127.0.0.1:1234/v1 \
  --verifier-model local-verifier-model \
  --ledger facts.json \
  --report audit.json \
  --fail-on-review
```

The verifier also must be non-Claude.

### Standard input and output

```bash
printf 'Authorized source text' | \
  claude-text-lab rewrite - -o - \
  --endpoint http://127.0.0.1:11434/v1 \
  --model qwen3:8b
```

For reproducible work, file output with `--ledger` and `--report` is preferable. Input, output, ledger, and report paths must be distinct.

### Benchmark many transformations

Each non-empty JSONL line contains a source and candidate:

```json
{"id":"case-001","source":"Original text","candidate":"Regenerated text","required_exact":["2026"]}
```

Run:

```bash
claude-text-lab benchmark examples/benchmark.jsonl \
  -o benchmark-report.json
```

Real observations from a future authoritative detector can be supplied in each record. Detector values are never guessed or generated by this project. See [docs/benchmark.md](docs/benchmark.md).

## Use a remote OpenAI-compatible endpoint

Loopback endpoints are allowed by default. A non-loopback endpoint must use HTTPS and requires explicit permission:

```bash
export MODEL_API_KEY='replace-with-a-real-secret'

claude-text-lab rewrite source.txt -o regenerated.txt \
  --endpoint https://models.example/v1 \
  --model non-claude-model \
  --api-key "$MODEL_API_KEY" \
  --allow-remote \
  --ledger facts.json \
  --report audit.json
```

Remote use sends the source, ledger, and candidates to that service. Check its retention, training, residency, and security terms first. The client rejects URL-embedded credentials, plain HTTP for remote hosts, and unsafe redirects.

Environment variables are available for repeatable local configuration:

| Variable | Use |
|---|---|
| `CTL_ENDPOINT` | Writer endpoint |
| `CTL_MODEL` | Writer model |
| `CTL_API_KEY` | Writer key |
| `CTL_VERIFIER_ENDPOINT` | CLI or MCP verifier endpoint |
| `CTL_VERIFIER_MODEL` | CLI or MCP verifier model |
| `CTL_VERIFIER_API_KEY` | CLI or MCP verifier key |
| `CTL_ALLOW_REMOTE` | MCP-only remote opt-in, accepted values include `true`, `1`, and `yes` |

CLI options take precedence where they exist. Do not commit keys to source control.

## MCP tutorial

MCP is useful when another application needs to call the inspector, sanitizer, auditor, or ledger-based rewriter as structured tools. It does not improve the transformation technique and does not add a Claude detector.

Install and test the server command:

```bash
python -m pip install '.[mcp]'
claude-text-lab-mcp
```

The server uses standard input and output, so a terminal appears to wait. That is normal. Press `Ctrl+C` and let an MCP host launch it instead.

It exposes four tools:

- `inspect_text_provenance`
- `sanitize_text_unicode`
- `audit_text_transformation`
- `regenerate_from_fact_ledger`

### Claude hosts are not supported

The server refuses every tool call from a Claude MCP host, including Claude Code and Claude Desktop. It reads `clientInfo` during initialization and returns an error when the reported host name or title contains `claude` or `anthropic`:

```text
Claude Text Lab does not support Claude MCP hosts, including Claude Code and Claude
Desktop. Run the command-line interface instead.
```

The reason is the transformation itself. A Claude host receives every tool result into a Claude conversation, and Anthropic states that supported Claude output can carry a mark. Returning a non-Claude candidate into that conversation defeats the work that produced it. Use the CLI, which writes the candidate straight to disk:

```bash
claude-text-lab rewrite source.txt -o regenerated.txt --report audit.json
```

Non-Claude MCP hosts are unaffected. VS Code, Cursor, Cline, Roo Code, OpenCode, Windsurf, Zed, and JetBrains all connect normally.

### Use a generic MCP JSON configuration

MCP clients and the MCP Inspector commonly accept an `mcpServers` configuration:

```json
{
  "mcpServers": {
    "claude-text-lab": {
      "type": "stdio",
      "command": "/absolute/path/to/project/.venv/bin/claude-text-lab-mcp",
      "env": {
        "CTL_ENDPOINT": "http://127.0.0.1:11434/v1",
        "CTL_MODEL": "qwen3:8b",
        "CTL_API_KEY": "local"
      }
    }
  }
}
```

Windows typically uses an executable path such as `C:\\absolute\\path\\.venv\\Scripts\\claude-text-lab-mcp.exe`. JSON backslashes must be doubled.

The full setup, tool argument examples, Inspector workflow, client caveats, and troubleshooting are in [docs/mcp.md](docs/mcp.md).

### Why the Claude-host refusal exists

If Claude were the MCP host, Claude could display, summarize, or rewrite the tool result after the non-Claude model returned it. Anthropic says supported Claude processing can carry a mark. Rather than leave that to a documentation warning, the server refuses those hosts outright.

The same reasoning applies to any AI host that paraphrases results into chat. When the exact non-Claude output matters, run the CLI and use `regenerated.txt` from disk rather than text an assistant retyped.

## Python API

Inspection and audit require no optional packages:

```python
from claude_text_lab import audit_texts, inspect_text, sanitize_unicode

source = "Example\u200b text from an authorized document."
cleaned = sanitize_unicode(source)
inspection = inspect_text(cleaned)
comparison = audit_texts(cleaned, "An authorized document contains an example.")

print(inspection.to_dict())
print(comparison.to_dict())
```

Run the full rewrite pipeline with any compatible client:

```python
from claude_text_lab.core import RewriteConfig, rewrite_text
from claude_text_lab.llm import OpenAICompatibleClient

client = OpenAICompatibleClient(
    endpoint="http://127.0.0.1:11434/v1",
    model="qwen3:8b",
    api_key="local",
)

config = RewriteConfig(
    style="concise technical documentation",
    candidates=3,
    review_ledger=True,
    semantic_check=True,
)

result = rewrite_text(client, "Authorized source text.", config, verifier=client)
print(result.text)
print(result.to_dict(include_text=False))
```

The Python API does not automatically refuse a Claude-named custom client. The packaged CLI and MCP wrapper perform that policy check. Application developers must enforce an equivalent non-Claude model policy themselves.

## Exit codes

| Code | Meaning |
|---:|---|
| `0` | Command completed. For `audit`, still inspect `quality_gate.passed`. |
| `2` | Invalid input, unsafe endpoint, file collision, model error, or other handled operation error. |
| `3` | `rewrite --fail-on-review` generated output, but no candidate passed all configured gates. |

## Troubleshooting

### `connection refused`

Start the local model server, confirm the port, and query its model list. For Ollama:

```bash
ollama list
curl http://127.0.0.1:11434/v1/models
```

### `model endpoint returned HTTP 404`

Use the provider's OpenAI-compatible base URL. Claude Text Lab appends `/chat/completions` unless the endpoint already ends with it. Do not pass Ollama's native `/api/chat` URL.

### `model did not return valid JSON`

Ledger extraction and verification require JSON. Retry with a stronger instruction-following model, reduce the source length, or use a provider with structured-output reliability. Do not treat repeated parser failures as successful transformation.

### `remote model endpoint refused`

Use a loopback address, or use an HTTPS remote endpoint with `--allow-remote`. Plain remote HTTP is intentionally rejected.

### `rewrite model must be non-Claude`

Select a non-Claude writer and verifier. The restriction is deliberate because Claude could add a new mark.

### Output file already exists

Choose a new path or add `--force`. The tool refuses silent replacement and refuses to make an output path equal to an input path.

### The candidate keeps failing overlap gates

Try a more capable model, use three or more candidates, change the requested style, or lower `--max-shared-run` only after examining why text is shared. Raising `--max-ngram-reuse` makes the gate more permissive, but it does not make the transformation stronger.

### The candidate loses facts

Use a stronger model, keep ledger review and semantic checking enabled, use a separate verifier, and examine `facts.json`. Add an exact value with `audit --require` when the deterministic patterns do not protect it. Human review remains required for high-stakes text.

### MCP server connects but rewrite fails

The MCP host launches a fresh process and may not inherit the shell environment you expected. Put `CTL_ENDPOINT`, `CTL_MODEL`, and `CTL_API_KEY` in the server configuration. Use an absolute executable path and check the host's MCP logs.

## Security and responsible use

Use Claude Text Lab only on text you own or are authorized to edit. Do not use it to evade disclosure requirements, academic-integrity rules, publishing policies, contractual duties, platform rules, or applicable law. A changed signal does not make AI-assisted work human-authored.

The source is treated as untrusted data in prompts, but prompt-injection risk cannot be eliminated. A local model server is also a trust boundary. Review [docs/threat-model.md](docs/threat-model.md) before processing sensitive or high-stakes material.

## Research basis and competing projects

The project was designed from Anthropic's August 2026 announcement, published text-watermark research, and a review of existing open-source tools. The detailed claim matrix, evidence tiers, method comparison, unknowns, experiment design, and limitations are in [docs/research.md](docs/research.md).

The linked projects provide useful Unicode, media, evaluation, or open-watermark ideas, but none contains Anthropic's undisclosed detector:

- [guillaumemeyer/watermarks-remover](https://github.com/guillaumemeyer/watermarks-remover)
- [wiltodelta/remove-ai-watermarks](https://github.com/wiltodelta/remove-ai-watermarks)
- [THU-BPM/MarkLLM](https://github.com/THU-BPM/MarkLLM)
- [jwkirchenbauer/lm-watermarking](https://github.com/jwkirchenbauer/lm-watermarking)
- [allencheng97/Self-information-Rewrite-Attack](https://github.com/allencheng97/Self-information-Rewrite-Attack)

No code from those repositories is copied or vendored here.

## Documentation map

- [Research note and comparison](docs/research.md)
- [Threat model and limitations](docs/threat-model.md)
- [Benchmark protocol](docs/benchmark.md)
- [MCP setup and tool tutorial](docs/mcp.md)
- [Examples](examples/README.md)
- [Release checklist](docs/releasing.md)
- [Contributing](CONTRIBUTING.md)
- [Security policy](SECURITY.md)

## Development and verification

```bash
python -m unittest discover -s tests -v
python -m compileall -q src tests
PYTHONPATH=src python -m claude_text_lab --version
PYTHONPATH=src python -m claude_text_lab --help
```

With the development extra installed:

```bash
python -m pytest
python -m build
python -m twine check dist/*
```

## License and independence

MIT. See [LICENSE](LICENSE).

Claude is a trademark of Anthropic. Claude Text Lab is an independent research project and is not affiliated with or endorsed by Anthropic.