Binoculars Local MCP

This repository is a local, security-focused derivative of Binoculars, the zero-shot AI-text detection method introduced in Spotting LLMs With Binoculars. The upstream authors created the scoring method and original implementation. This project adds a read-only MCP interface, offline model loading, process-level network blocking, a compact local model profile, and profile-specific calibration.

The detector returns a statistical signal, not proof that a person or model wrote a passage. It should not be the sole basis for grading, discipline, employment decisions, accusations, or other consequential action.

Local security boundary

• MCP transport is stdio; the server does not listen on a port.

• Before MCP, model, or HTTP-capable dependencies import, the MCP process forces Hugging Face and Transformers offline modes and installs a guard for common Python TCP/UDP socket egress. Local Unix-domain sockets remain available to libraries.

• Model arguments must resolve to existing local directories; remote model IDs are rejected.

• Loads require local_files_only=True, trust_remote_code=False, and SafeTensors weights.

• The compact profile verifies SHA-256 digests for every required config, tokenizer, vocabulary, merge, generation-config, and SafeTensors file before reporting readiness or loading the detector. Cached digests are invalidated when file identity or metadata changes.

• Both tools are read-only and declare openWorldHint=false.

• Input is limited to 100,000 characters, and the compact profile analyzes at most 256 tokens.

Installing Python packages may contact a package index. Model acquisition is a separate explicit step. Once configured, model inference uses only verified local files. The in-process guard covers common Python socket APIs; it is defense in depth, not an operating-system sandbox or a guarantee against native code, subprocesses, or a compromised dependency. Use an OS-level network sandbox when that stronger boundary is required.

Related MCP server: avoid-ai-writing-mcp

Install

Python 3.11 through 3.13 is supported. This checkout uses Python 3.12 with uv:

On a Mac, begin with the optional setup helper. Its default mode only reports macOS, processor, memory, disk, Python, and uv readiness:

./scripts/setup_macos.sh

Dependency installation and model acquisition remain separate choices:

./scripts/setup_macos.sh --install
./scripts/setup_macos.sh --download-models
source .models/env.zsh

The second command explains the download before it starts, fetches the two pinned model revisions, verifies every file in each pinned model manifest, and writes an ignored local environment file. Use --yes only after reviewing the requested operation.

For manual setup:

uv sync --python python3.12 --extra dev --locked

Model weights stay outside Git under .models/. Point the server at compatible local directories:

export BINOCULARS_PROFILE=qwen2.5-0.5b
export BINOCULARS_OBSERVER_MODEL=/path/to/qwen2.5-0.5b-base
export BINOCULARS_PERFORMER_MODEL=/path/to/qwen2.5-0.5b-instruct
export BINOCULARS_DEVICE=cpu  # or mps, cuda, cuda:N, or auto

Without both model directories, the MCP server can still start, but binoculars_status reports that analysis is not ready. Configuration is captured once when the MCP process starts; restart it after changing these environment variables.

Compact model profile

The qwen2.5-0.5b profile pairs Qwen2.5-0.5B with Qwen2.5-0.5B-Instruct. Their SafeTensors weights use about 1.9 GB together, substantially less than the two 7B models in the original configuration. Repository revisions, file hashes, thresholds, token limit, and the calibration summary are pinned in binoculars/profiles.py.

This compact profile was calibrated on 360 local texts and evaluated on a separate 240-text holdout derived from CC-News, CNN, and PubMed human/Falcon samples. The calibration corpus is not distributed with this repository. On that holdout, accuracy mode reached 88.3%. Conservative mode measured a 2.5% human false-positive rate and a 60.0% machine true-positive rate. This is a limited local benchmark, not a general performance guarantee and not the original paper's evaluation.

MCP tools

binoculars_status

Reports model readiness, selected profile, weight-hash checks, stdio transport, the active network-guard probe, and calibration context. Call it before analysis.

binoculars_analyze_text

Accepts text and an optional threshold mode:

• low-fpr is the default and is more conservative about AI-generated labels.

• accuracy provides the profile's broader screening threshold.

The result includes the score, threshold, mode, label, model profile, device, character count, original tokenizer count, analyzed tokenizer count, and whether truncation occurred. Lower scores indicate a stronger AI-generated signal under the configured threshold. Inputs with fewer than eight analyzed tokens are rejected instead of labeled, as are non-finite numerical results. Samples of roughly 200-300 words are generally more useful than very short passages.

Run and verify

Start the stdio server:

.venv/bin/binoculars-mcp

Available project checks are:

.venv/bin/pytest -q
.venv/bin/ruff check binoculars tests
.venv/bin/bandit -q -c pyproject.toml -r binoculars
.venv/bin/pip-audit

Requests wait at most 120 seconds for another analysis to release the detector lock. Set BINOCULARS_LOCK_TIMEOUT_SECONDS between 0.1 and 3600 to change that queueing bound. Once native model inference starts, MCP cancellation cannot safely interrupt it in-process; process isolation is required for a hard execution deadline.

The default test suite uses tiny local models and runs without downloads. With the pinned models already present, run the CPU/MPS golden and performance checks explicitly:

BINOCULARS_RUN_DEVICE_TESTS=1 \
BINOCULARS_OBSERVER_MODEL=/path/to/qwen2.5-0.5b-base \
BINOCULARS_PERFORMER_MODEL=/path/to/qwen2.5-0.5b-instruct \
uv run pytest -q tests/test_device_regression.py

Inspect a Codex registration with:

codex mcp get binoculars-local
codex mcp list

Inference for a local stdio server uses the machine's MPS or CPU resources. It does not use hosted Codex inference compute.

Local performance

Measurements below were taken on a 24 GB M4 Pro MacBook Pro with macOS 26.6.2. Each run loaded both pinned Qwen2.5-0.5B models and analyzed the same 1,196-character passage six times.

The first uncached MPS process took 2.85 seconds to load the models and 2.21 seconds for its first call. Later runs benefited from operating-system file caches. Reproduce the measurement with:

uv run python scripts/benchmark_local.py --device mps --iterations 6
uv run python scripts/benchmark_local.py --device cpu --iterations 6

Limitations

Binoculars signals are weaker on short, non-English, heavily quoted, or memorized text. Results also depend on the selected observer/performer models and calibration data. Report the score, threshold, mode, and profile together; do not translate one label into certainty about authorship.

Attribution

The Binoculars score and original implementation are by Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. See the paper, upstream repository, and preserved license.

The local MCP runtime, offline and network guards, compact profile configuration, weight verification, security tests, and local calibration are Gaurav Dama's derivative work. The underlying detection method remains the upstream authors' work.