Skip to main content
Glama

Chopper 0.3B close to Opus 5.5 in cutting out the best pieces

Chops are pieces cut out of the original text snippets retrieved from a database or a long conversation: exact quotes, never rewritten. The chopper (mmBERT, 0.31B) makes them; for comparison, Claude Opus 5.5 makes them too, also forced to quote directly.

What is new: a new method, invented in November 2026 in Poland (EU). The chopper is not an LLM summarising the snippets: a small encoder reads the question and all 20 snippets together in one pass and marks every word keep or drop. There is no text generation, so it is fast: 50 ms per question (20 snippets, ~2,400 tokens) on a consumer-grade GPU (RTX 3090), 36 ms with mmBERT-small.

One database retrieval per question, even at 4 hops. Every result below uses a single retrieval: each question gets its 20 snippets once, the chopper cuts them once, and Qwen3-14B answers once. There are no follow-up searches and no second rounds, so a 4-hop question has to be chained from one cut.

A reranker that also gives you the chops

One model replaces both the reranker and the compressor. Full comparison: below.

Qwen3-14B answering from full snippets or just short chops

400 MuSiQue questions, 100 each at 1, 2, 3 and 4 hops, each with its own 20 retrieved snippets. Qwen3-14B (4-bit, no fine-tuning, no thinking mode) answers from the given text only; gpt-5.6-luna judges. Length = prompt tokens. Each score counts every correct answer, including those from incomplete chops; the number in brackets counts only questions whose chops kept every hop and the answer.

Qwen3-14B: all snippets vs chops from the chopper vs chops from Opus 5.5

Read each diagram like website traffic: questions flow left to right, and at every hop the ones whose fact the chops missed drop out; the last stage is Qwen3-14B answering correctly. Top: all 20 retrieved snippets. Middle: the chopper's chops. Bottom: Opus 5.5's chops.

The last stage is the table's score; the lighter block under it counts answers Qwen got right from incomplete chops, and the number in brackets leaves them out. More: docs/DETAILS.md ยง7.

Also a reranker: ranking and chopping in one pass

The chopper scores every passage while it chops (mean p(KEEP) of the passage's 16 most likely tokens). On the 8-language rerank holdout (400 queries, 16 passages each, never trained on), ranked against dedicated rerankers:

ranking 16 passages

nDCG@10

Hit@1

GPU time per query (RTX 3090)

chopper, mmBERT-base (0.31B), one pass

0.973

0.972

0.20 s, chops + ANSWER/SEARCH included

bge-reranker-v2-m3 (0.57B)

0.984

0.985

0.20 s

Qwen3-Reranker-0.6B

0.977

0.972

0.77 s

Qwen3-Reranker-4B

0.986

0.990

3.3 s

The chopper was never trained to rank, yet it is within about one point of the best rerankers at the speed of the fastest, so a pipeline can drop its reranker: retrieve โ†’ chopper gives the ranking and the quotes. Caveats (this test is easy for every model; the labels favour Qwen3-Reranker-0.6B) in the details.

Against other context compressors

Same 400 questions, same 20 snippets, same Qwen3-14B prompt; each compressor cuts the snippets once:

Qwen3-14B answering from

tokens

1-hop

2-hop

3-hop

4-hop

all 20 snippets (no compression)

2,620

84

28

22

12

LLMLingua-2 (Microsoft), rate 0.1

374

28

6

2

4

Provence (Naver), its default threshold 0.1

263

77

27

26

16

Provence, threshold 0.5

152

71

24

21

11

chops from the chopper, mmBERT-small (0.14B)

217

72

49

32

23

chops from the chopper, mmBERT-base (0.31B)

228

83

48

45

29

LLMLingua-2 drops tokens without looking at the question, so the facts a chain needs rarely survive. Provence reads the question but prunes each snippet on its own. To be fair to them: the chopper was trained on MuSiQue's train split (this test uses its dev split) and on other multi-hop data, while both were trained for general single-passage use.


Chopper ๐Ÿ”ช keep only the part of RAG that answers the question

Standard RAG (retrieval-augmented generation) pastes whole passages into your model's context, and they stay there. About 95% of those tokens don't help. After a few lookups, a long session is mostly retrieval spam.

Chopper is a small model that reads the retrieved passages first and keeps only the fragments that matter. Your model then sees ~100 tokens instead of ~3,000 per lookup, and a session can carry dozens of lookups.

question โ”€โ–บ retrieve 16 passages โ”€โ–บ ๐Ÿ”ช rank + chop in one pass โ”€โ–บ model answers, or asks one follow-up search
  • ๐ŸŽฏ Clean context: only useful fragments enter the session; the raw passages never stay.

  • โšก Less compute: the main model reads 2โ€“5% of the retrieved text.

  • ๐Ÿ… Also a reranker: the same pass ranks the passages almost as well as bge-reranker-v2-m3 (nDCG@10 0.973 vs 0.984, 8 languages), at the same speed. No separate reranker needed.

  • ๐Ÿ”— Multi-hop works: the model writes a follow-up question when a fact is missing, and searches again.

  • ๐Ÿ  Built on one home GPU: every dataset and model here was made on a single RTX 3090 (some labels came from APIs).

Related MCP server: gus

Results at a glance

retrieved text the main model reads

2โ€“5% (about 20โ€“40ร— less); the answer is still in the kept text for 85% of HotpotQA and 69% of TriviaQA questions

"nothing here": pools whose answer was deleted (8 languages, business documents)

asks to search again 67% of the time, keeps 1.4% of the text

ranking 16 passages (8-language holdout, 400 queries)

nDCG@10 0.973, vs bge-reranker-v2-m3 0.984 and Qwen3-Reranker-4B 0.986; 0.20 s like bge, 16ร— faster than Qwen-4B, chops included (details)

chopping time per lookup (RTX 3090)

155 ms (separate mmBERT) ยท 277 ms (built into MiniCPM3-4B)

4-hop MuSiQue questions answered correctly by Qwen3-14B, one retrieval

12% โ†’ 29% with the chopper's chops instead of all 20 snippets (11ร— fewer tokens); with Opus 5.5's chops: 43%

Polish questions (PoQuAD, PolQA) answered correctly by Qwen3-14B

85.6% from the Polish chopper's chops (255 tokens) vs 85.2% from all 16 snippets (3,856 tokens)

license

MIT: free for commercial use, weights and code. The closest alternatives, Provence and XProvence (Naver, sentence pruning + reranking), are CC BY-NC-ND 4.0: non-commercial only

In training (Nov 2026), suggested by the OPI PIB AI Lab: the same Polish chopper recipe on two Polish encoders, pl-ModernBERT-base from OPI PIB (results expected about 16 Nov) and polish-roberta-base-8k (8k context, about 18 Nov), compared with mmBERT-base-pl on the same 856 Polish questions. Restarted 15 Nov with the training data fitted to the Polish tokenizers (why).

Long-conversation memory results (LoCoMo, LongMemEval) are in the details: there the 4B answering model is the limit.

Pick your chopper

๐ŸŸง universal

๐ŸŸช integrated

what it is

a separate small encoder (mmBERT-base, 0.31B, recommended; or mmBERT-small, 0.14B; both also in Polish versions)

a 20% slice of MiniCPM3-4B's own weights + a small LoRA

works with

any answering model: text in, text out

MiniCPM3-4B

extra memory

0.31B / 0.14B weights

45M parameters; reads the model's own cache

weights

๐Ÿค— wiiiktor/chopper_mmbert

๐Ÿค— wiiiktor/chopper_minicpm3

Start with the universal one. The integrated one is the long-term bet: trained together with a model from the start, it should become almost free. If you pretrain models and want to build that, contact wiiiktor.

Quick start

No install: try the in-browser demo (po polsku). It runs small (0.14B); the code below uses the more accurate base (0.31B).

In your own RAG pipeline (LangChain, LlamaIndex, Haystack or plain Python):

pip install chopper-rag                  # or "chopper-rag[langchain]", [llama-index], [haystack], [all]

As a tool for Claude (Claude Code, Claude Desktop or any MCP client): pip install "chopper-rag[mcp]", then claude mcp add chopper -- chopper-mcp. Claude can then call chop_passages / chop_files to read a few quotes instead of whole documents, or rank_passages to rerank; setup in integrations/chopper-rag.

from chopper_rag import Chopper
chopper = Chopper.from_pretrained(variant="base")          # or "small"; for Polish "base-pl", "small-pl"
result = chopper.run("When was the Eiffel Tower completed?", passages)   # passages: list of retrieved texts
print([c.text for c in result.chops], result.decision)    # exact quotes + ANSWER / SEARCH
print(result.ranking(top_k=5))                             # the same pass as a reranker: [(passage_index, relevance), ...]

Framework examples: integrations/chopper-rag. Listed in the LangChain integrations (ChopperCompressor). Or try the demo in this repo:

pip install -r requirements.txt

# your documents (a folder of .txt / .md, or a .jsonl with "title" and "text") + any Hugging Face causal LM
python operator-for-llm-with-mmbert.py --docs my_docs/ --model <hf model id> \
       --ask "Who founded the company that distributed the film UHF?"

# leave out --ask for an interactive session; it shows how many tokens plain RAG would have added

Weights download from Hugging Face on first use. Training data: ๐Ÿค— wiiiktor/chopper_demo.

Want the details?

Everything else is on one page, docs/DETAILS.md:

Credits

Built on mmBERT (JHU CLSP), MiniCPM3-4B (OpenBMB), Qwen3, Qwen3-Embedding and Qwen3-Reranker (Qwen team), bge-reranker (BAAI); labels and judgements also came from Claude Opus 5.5 (Anthropic) and gpt-5.6-luna / GPT-6 Luna (OpenAI). Compared with LLMLingua-2 (Microsoft) and Provence (Naver). Data: the MuSiQue, HotpotQA, 2WikiMultiHopQA, TriviaQA, QASPER, Multi-Session Chat, WildChat, FineWeb, Wikipedia, LoCoMo and LongMemEval datasets; for Polish, PoQuAD and PolQA (IPI PAN). Our memory evaluation follows Memory Tax (He et al., 2026) and compares with MemT-1.0 (MemTensor, 2026). Full table with licenses: docs/DETAILS.md.

Licenses

Commercial use is allowed. The mmBERT choppers and all code are MIT. For comparison, Naver's Provence and XProvence context pruners are licensed CC BY-NC-ND 4.0 (no commercial use, no derivatives).

what

license

mmBERT choppers, incl. Polish (mmbert-base, mmbert-small, mmbert-base-pl, mmbert-small-pl)

MIT (as mmBERT)

code in this repo (LICENSE) and pip install chopper-rag

MIT

MiniCPM3 chopper

MiniCPM Model License (as MiniCPM3-4B)

training data wiiiktor/chopper_demo

CC BY-SA 4.0 (share-alike from HotpotQA, Wikipedia and PolQA); Polish parts: PoQuAD CC BY 4.0, PolQA CC BY-SA 4.0

Some labels came from Claude Opus 5.5 and OpenAI Luna models, used to train a non-competing extraction tool; see docs/DETAILS.md.


Bored with reading LLMs' outputs? See also my other project, control_room: a VS Code panel for Claude Code that shows each answer as cards, charts and clickable options instead of a wall of text.

Available Tools

3 tools
chop_filesA

Read local text files (notes, docs, logs, code, exported pages), split them into ~1,000-character passages and cut out the exact quotes that answer question. Each quote carries its file path and character offset in that file. Use it instead of reading long files in full when only part of them is relevant.

Inputs: paths (required), question (required).

chop_passagesA

Cut the exact quotes that answer question out of passages (retrieved snippets, search results, tool outputs). Returns the quotes (verbatim, with passage number and character offsets), a ready-to-read context string, and decision ANSWER (the passages hold the answer) or SEARCH (search again).

Inputs: passages (required), question (required).

rank_passagesB

Rank passages by relevance to question (the chopper used as a reranker: on a held-out 8-language set it ranks within ~1 nDCG point of bge-reranker-v2-m3 at the same speed). Returns the top_k passages, most relevant first, with their index and relevance (0..1), plus decision ANSWER / SEARCH.

Inputs: passages (required), question (required), top_k.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelogโ€ฆ

    TDQS

    A3.8/5.0

    Scored across 3 tools

    Disambiguation4/5

    rank_passages is clearly distinct, and chop_passages vs chop_files differ by input source. However, the two chop tools share the same core extraction behavior and return shape, so an agent could briefly hesitate between them when file-derived snippets are involved.

    Naming Consistency5/5

    All names use consistent snake_case verb_noun form: chop_passages, rank_passages, chop_files. The shared chop_ prefix accurately groups related operations without introducing inconsistent conventions.

    Tool Count5/5

    Three tools is a tight, well-scoped surface for a quote-extraction/reranking utility. Each tool covers a distinct input mode or operation, with no filler tools.

    Completeness5/5

    The surface covers both core inputs (provided passages and local files) plus reranking, and returns decisions that guide next steps. No obvious operation is missing for the stated chopper/reranker domain.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      Not graded
      maintenance
      Enables AI agents to compress retrieved web documents against a query, pruning irrelevant noise so only the most relevant context reaches the model. It runs as a zero-dependency Python MCP server that accepts a query payload and returns structured, scored output.
      -
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI coding agents to retrieve only the specific code sections that answer their questions, drastically reducing context usage while running fully locally.
      1
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Provides local-first retrieval over long design conversations, enabling users to find where a decision was made, why, and what replaced it, with turn and line range citations through hybrid retrieval, typed judgment rerank, exact search, and five MCP tools.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Ranks and filters retrieved context locally on your machine to reduce LLM token usage, cutting context tokens by ~67% while preserving relevant passages.
      55 npm
      1
      MIT