Chopper
Chopper 0.3B close to Opus 5.5 in cutting out the best pieces
Chops are pieces cut out of the original text snippets retrieved from a database or a long conversation: exact quotes, never rewritten. The chopper (mmBERT, 0.31B) makes them; for comparison, Claude Opus 5.5 makes them too, also forced to quote directly.
What is new: a new method, invented in November 2026 in Poland (EU). The chopper is not an LLM summarising the snippets: a small encoder reads the question and all 20 snippets together in one pass and marks every word keep or drop. There is no text generation, so it is fast: 50 ms per question (20 snippets, ~2,400 tokens) on a consumer-grade GPU (RTX 3090), 36 ms with mmBERT-small.
One database retrieval per question, even at 4 hops. Every result below uses a single retrieval: each question gets its 20 snippets once, the chopper cuts them once, and Qwen3-14B answers once. There are no follow-up searches and no second rounds, so a 4-hop question has to be chained from one cut.
A reranker that also gives you the chops
One model replaces both the reranker and the compressor. Full comparison: below.
Qwen3-14B answering from full snippets or just short chops
400 MuSiQue questions, 100 each at 1, 2, 3 and 4 hops, each with its own 20 retrieved snippets. Qwen3-14B (4-bit, no fine-tuning, no thinking mode) answers from the given text only; gpt-5.6-luna judges. Length = prompt tokens. Each score counts every correct answer, including those from incomplete chops; the number in brackets counts only questions whose chops kept every hop and the answer.
Qwen3-14B: all snippets vs chops from the chopper vs chops from Opus 5.5
Read each diagram like website traffic: questions flow left to right, and at every hop the ones whose fact the chops missed drop out; the last stage is Qwen3-14B answering correctly. Top: all 20 retrieved snippets. Middle: the chopper's chops. Bottom: Opus 5.5's chops.
The last stage is the table's score; the lighter block under it counts answers Qwen got right from incomplete chops, and the number in brackets leaves them out. More: docs/DETAILS.md ยง7.
Also a reranker: ranking and chopping in one pass
The chopper scores every passage while it chops (mean p(KEEP) of the passage's 16 most likely tokens). On the 8-language rerank holdout (400 queries, 16 passages each, never trained on), ranked against dedicated rerankers:
ranking 16 passages | nDCG@10 | Hit@1 | GPU time per query (RTX 3090) |
chopper, mmBERT-base (0.31B), one pass | 0.973 | 0.972 | 0.20 s, chops + ANSWER/SEARCH included |
bge-reranker-v2-m3 (0.57B) | 0.984 | 0.985 | 0.20 s |
0.977 | 0.972 | 0.77 s | |
0.986 | 0.990 | 3.3 s |
The chopper was never trained to rank, yet it is within about one point of the best rerankers at the speed of the fastest,
so a pipeline can drop its reranker: retrieve โ chopper gives the ranking and the quotes. Caveats (this test is easy
for every model; the labels favour Qwen3-Reranker-0.6B) in the details.
Against other context compressors
Same 400 questions, same 20 snippets, same Qwen3-14B prompt; each compressor cuts the snippets once:
Qwen3-14B answering from | tokens | 1-hop | 2-hop | 3-hop | 4-hop |
all 20 snippets (no compression) | 2,620 | 84 | 28 | 22 | 12 |
LLMLingua-2 (Microsoft), rate 0.1 | 374 | 28 | 6 | 2 | 4 |
Provence (Naver), its default threshold 0.1 | 263 | 77 | 27 | 26 | 16 |
Provence, threshold 0.5 | 152 | 71 | 24 | 21 | 11 |
chops from the chopper, mmBERT-small (0.14B) | 217 | 72 | 49 | 32 | 23 |
chops from the chopper, mmBERT-base (0.31B) | 228 | 83 | 48 | 45 | 29 |
LLMLingua-2 drops tokens without looking at the question, so the facts a chain needs rarely survive. Provence reads the question but prunes each snippet on its own. To be fair to them: the chopper was trained on MuSiQue's train split (this test uses its dev split) and on other multi-hop data, while both were trained for general single-passage use.
Chopper ๐ช keep only the part of RAG that answers the question
Standard RAG (retrieval-augmented generation) pastes whole passages into your model's context, and they stay there. About 95% of those tokens don't help. After a few lookups, a long session is mostly retrieval spam.
Chopper is a small model that reads the retrieved passages first and keeps only the fragments that matter. Your model then sees ~100 tokens instead of ~3,000 per lookup, and a session can carry dozens of lookups.
question โโบ retrieve 16 passages โโบ ๐ช rank + chop in one pass โโบ model answers, or asks one follow-up search๐ฏ Clean context: only useful fragments enter the session; the raw passages never stay.
โก Less compute: the main model reads 2โ5% of the retrieved text.
๐ Also a reranker: the same pass ranks the passages almost as well as bge-reranker-v2-m3 (nDCG@10 0.973 vs 0.984, 8 languages), at the same speed. No separate reranker needed.
๐ Multi-hop works: the model writes a follow-up question when a fact is missing, and searches again.
๐ Built on one home GPU: every dataset and model here was made on a single RTX 3090 (some labels came from APIs).
Related MCP server: gus
Results at a glance
retrieved text the main model reads | 2โ5% (about 20โ40ร less); the answer is still in the kept text for 85% of HotpotQA and 69% of TriviaQA questions |
"nothing here": pools whose answer was deleted (8 languages, business documents) | asks to search again 67% of the time, keeps 1.4% of the text |
ranking 16 passages (8-language holdout, 400 queries) | nDCG@10 0.973, vs bge-reranker-v2-m3 0.984 and Qwen3-Reranker-4B 0.986; 0.20 s like bge, 16ร faster than Qwen-4B, chops included (details) |
chopping time per lookup (RTX 3090) | 155 ms (separate mmBERT) ยท 277 ms (built into MiniCPM3-4B) |
4-hop MuSiQue questions answered correctly by Qwen3-14B, one retrieval | 12% โ 29% with the chopper's chops instead of all 20 snippets (11ร fewer tokens); with Opus 5.5's chops: 43% |
Polish questions (PoQuAD, PolQA) answered correctly by Qwen3-14B | 85.6% from the Polish chopper's chops (255 tokens) vs 85.2% from all 16 snippets (3,856 tokens) |
license | MIT: free for commercial use, weights and code. The closest alternatives, Provence and XProvence (Naver, sentence pruning + reranking), are CC BY-NC-ND 4.0: non-commercial only |
In training (Nov 2026), suggested by the OPI PIB AI Lab: the same Polish chopper recipe on two Polish encoders, pl-ModernBERT-base from OPI PIB (results expected about 16 Nov) and polish-roberta-base-8k (8k context, about 18 Nov), compared with mmBERT-base-pl on the same 856 Polish questions. Restarted 15 Nov with the training data fitted to the Polish tokenizers (why).
Long-conversation memory results (LoCoMo, LongMemEval) are in the details: there the 4B answering model is the limit.
Pick your chopper
๐ง universal | ๐ช integrated | |
what it is | a separate small encoder (mmBERT-base, 0.31B, recommended; or mmBERT-small, 0.14B; both also in Polish versions) | a 20% slice of MiniCPM3-4B's own weights + a small LoRA |
works with | any answering model: text in, text out | MiniCPM3-4B |
extra memory | 0.31B / 0.14B weights | 45M parameters; reads the model's own cache |
weights |
Start with the universal one. The integrated one is the long-term bet: trained together with a model from the start, it should become almost free. If you pretrain models and want to build that, contact wiiiktor.
Quick start
No install: try the in-browser demo (po polsku). It runs small (0.14B); the code below uses the more accurate base (0.31B).
In your own RAG pipeline (LangChain, LlamaIndex, Haystack or plain Python):
pip install chopper-rag # or "chopper-rag[langchain]", [llama-index], [haystack], [all]As a tool for Claude (Claude Code, Claude Desktop or any MCP client): pip install "chopper-rag[mcp]", then
claude mcp add chopper -- chopper-mcp. Claude can then call chop_passages / chop_files to read a few quotes
instead of whole documents, or rank_passages to rerank; setup in integrations/chopper-rag.
from chopper_rag import Chopper
chopper = Chopper.from_pretrained(variant="base") # or "small"; for Polish "base-pl", "small-pl"
result = chopper.run("When was the Eiffel Tower completed?", passages) # passages: list of retrieved texts
print([c.text for c in result.chops], result.decision) # exact quotes + ANSWER / SEARCH
print(result.ranking(top_k=5)) # the same pass as a reranker: [(passage_index, relevance), ...]Framework examples: integrations/chopper-rag. Listed in the LangChain integrations (ChopperCompressor). Or try the demo in this repo:
pip install -r requirements.txt
# your documents (a folder of .txt / .md, or a .jsonl with "title" and "text") + any Hugging Face causal LM
python operator-for-llm-with-mmbert.py --docs my_docs/ --model <hf model id> \
--ask "Who founded the company that distributed the film UHF?"
# leave out --ask for an interactive session; it shows how many tokens plain RAG would have addedWeights download from Hugging Face on first use. Training data: ๐ค wiiiktor/chopper_demo.
Want the details?
Everything else is on one page, docs/DETAILS.md:
how it works: double prefill, rollback, the answer-or-search decision;
all results, including the search loop and the 8-language rerank pilot;
Credits
Built on mmBERT (JHU CLSP), MiniCPM3-4B (OpenBMB), Qwen3, Qwen3-Embedding and Qwen3-Reranker (Qwen team), bge-reranker (BAAI); labels and judgements also came from Claude Opus 5.5 (Anthropic) and gpt-5.6-luna / GPT-6 Luna (OpenAI). Compared with LLMLingua-2 (Microsoft) and Provence (Naver). Data: the MuSiQue, HotpotQA, 2WikiMultiHopQA, TriviaQA, QASPER, Multi-Session Chat, WildChat, FineWeb, Wikipedia, LoCoMo and LongMemEval datasets; for Polish, PoQuAD and PolQA (IPI PAN). Our memory evaluation follows Memory Tax (He et al., 2026) and compares with MemT-1.0 (MemTensor, 2026). Full table with licenses: docs/DETAILS.md.
Licenses
Commercial use is allowed. The mmBERT choppers and all code are MIT. For comparison, Naver's Provence and XProvence context pruners are licensed CC BY-NC-ND 4.0 (no commercial use, no derivatives).
what | license |
mmBERT choppers, incl. Polish ( | MIT (as mmBERT) |
code in this repo (LICENSE) and | MIT |
MiniCPM3 chopper | MiniCPM Model License (as MiniCPM3-4B) |
training data | CC BY-SA 4.0 (share-alike from HotpotQA, Wikipedia and PolQA); Polish parts: PoQuAD CC BY 4.0, PolQA CC BY-SA 4.0 |
Some labels came from Claude Opus 5.5 and OpenAI Luna models, used to train a non-competing extraction tool; see docs/DETAILS.md.
Bored with reading LLMs' outputs? See also my other project, control_room: a VS Code panel for Claude Code that shows each answer as cards, charts and clickable options instead of a wall of text.
Available Tools
3 toolschop_filesA
Read local text files (notes, docs, logs, code, exported pages), split them into ~1,000-character passages and
cut out the exact quotes that answer question. Each quote carries its file path and character offset in that
file. Use it instead of reading long files in full when only part of them is relevant.
Inputs: paths (required), question (required).
chop_passagesA
Cut the exact quotes that answer question out of passages (retrieved snippets, search results, tool
outputs). Returns the quotes (verbatim, with passage number and character offsets), a ready-to-read context
string, and decision ANSWER (the passages hold the answer) or SEARCH (search again).
Inputs: passages (required), question (required).
rank_passagesB
Rank passages by relevance to question (the chopper used as a reranker: on a held-out 8-language set it
ranks within ~1 nDCG point of bge-reranker-v2-m3 at the same speed). Returns the top_k passages, most relevant
first, with their index and relevance (0..1), plus decision ANSWER / SEARCH.
Inputs: passages (required), question (required), top_k.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelogโฆ
TDQS
Scored across 3 tools
rank_passages is clearly distinct, and chop_passages vs chop_files differ by input source. However, the two chop tools share the same core extraction behavior and return shape, so an agent could briefly hesitate between them when file-derived snippets are involved.
All names use consistent snake_case verb_noun form: chop_passages, rank_passages, chop_files. The shared chop_ prefix accurately groups related operations without introducing inconsistent conventions.
Three tools is a tight, well-scoped surface for a quote-extraction/reranking utility. Each tool covers a distinct input mode or operation, with no filler tools.
The surface covers both core inputs (provided passages and local files) plus reranking, and returns decisions that guide next steps. No obvious operation is missing for the stated chopper/reranker domain.
Maintenance
Related MCP Connectors
Extractive document summarizer: returns the text's own sentences, so it cannot add facts.
Exact source text, not summaries: check citations and quotes in one call; read pages and PDFs.
Source-traced evidence research for AI agents. We organise the evidence; you decide.
Source-backed prompting and coding-agent guidance with provenance and freshness checks.
Related MCP Servers
- FlicenseNot gradedqualityNot gradedmaintenanceEnables AI agents to compress retrieved web documents against a query, pruning irrelevant noise so only the most relevant context reaches the model. It runs as a zero-dependency Python MCP server that accepts a query payload and returns structured, scored output.-
- AlicenseNot gradedqualityBmaintenanceEnables AI coding agents to retrieve only the specific code sections that answer their questions, drastically reducing context usage while running fully locally.1MIT
- AlicenseNot gradedqualityCmaintenanceProvides local-first retrieval over long design conversations, enabling users to find where a decision was made, why, and what replaced it, with turn and line range citations through hybrid retrieval, typed judgment rerank, exact search, and five MCP tools.MIT

AsterMind MCPofficial
AlicenseNot gradedqualityBmaintenanceRanks and filters retrieved context locally on your machine to reduce LLM token usage, cutting context tokens by ~67% while preserving relevant passages.55 npm1MIT