Pocketpedia

Offline English Wikipedia for local LLMs: a 14 GB index on your own disk, a command line tool and an MCP server. Search runs locally (re-ranking uses a local embedding model by default), so nothing leaves your machine.

A 4B model with Pocketpedia beats a 120B model answering from memory. On the same 500 held-out PopQA questions, Gemma 3 4B with Pocketpedia scored 49.6% (95% CI 45.2–54.0). GPT-OSS 120B without it scored 34.4% (30.4–38.7), even on the scoring most favourable to it.

The gain is largest on obscure entities, the long tail a small model cannot memorise. On SimpleQA (300 questions), Gemma 3 4B goes from 3.3% to 40.3% correct.

How the numbers were made, and what they don't show:

• The plan and the bar for "good enough to publish" were written down before any results.

• Retrieval was tuned on 300 development questions. All headline numbers come from 1,000 different test questions (200 per popularity quintile, seed 20261031).

• A fresh clone reproduced the Gemma numbers exactly (bar check).

• GPT-OSS 120B and Qwen 3.8 27B ran on a stratified half (500 questions); the headline compares the same 500. Mistral Medium 3.5 128B could not be loaded and was not run.

• GPT-OSS often put narrow no-break spaces inside names. Its plain-match score was 19.8%; the 34.4% above is after Unicode normalisation, which was added after registration in its favour.

• PopQA is entity-centric and Wikipedia-native, close to the best case for this approach. SimpleQA was graded by a local GPT-OSS 120B with the official template, not the official grader, so it is not comparable with published SimpleQA scores.

Full tables and limitations: bench/results.md. Benchmark instructions: bench/README.md.

Quickstart

Requires Python 3.12 or later with SQLite FTS5. From a clone of this repository:

python3 -m venv .venv
. .venv/bin/activate
pip install -e '.[build]'
pocketpedia download
pocketpedia build
pocketpedia doctor
pocketpedia search "Who directed Pixels" --retrieval v1
pocketpedia read "Pixels (2010 film)" --max-chars 3000

The download is the 41 English wikimedia/wikipedia 20231101.en parquet shards, about 11.6 GB. Building took about five minutes on the benchmark machine; allow roughly 25 GB for the download and indexes, plus room for caches (the benchmark directory reached about 26 GB). Build time depends on your CPU and disk. No Wikipedia text or index is shipped with this package.

Downloads run three shards at a time. Re-running resumes .part files and skips completed shards. Use an empty download directory when switching from the old shell downloader, which did not distinguish partial from complete files. build refuses to overwrite an existing index; choose a new --out to rebuild.

Data defaults to ~/.pocketpedia. --data-dir overrides $POCKETPEDIA_HOME, which overrides that default. Options work before or after the command.

pocketpedia download --dir /path/to/parquet
pocketpedia build --parquet /path/to/parquet --out /path/to/data/index
pocketpedia search "Who directed Pixels" -k 5 --json --retrieval v1 --data-dir /path/to/data
pocketpedia doctor --data-dir /path/to/data

index/index.sqlite holds titles and leads; index/fulltext.sqlite holds compressed full articles. Read and v1 search need no service. PyArrow is imported only by build; pip install -e . is enough to use an existing index. Python 3.14 uses stdlib zstd; Python 3.12–3.13 install zstandard automatically.

Related MCP server: mcp-offline-wikipedia

Retrieval

V1 is the benchmark's boosted retrieval: the longest question span matching an article title goes first, followed by SQLite FTS5 BM25 results over titles and leads. A lead is the first two nonempty paragraphs, capped at 1,500 characters. V2 adds parenthesised title candidates, uses embeddings to select five articles, splits their first 8,000 words into roughly 120–200-word passages, then reranks the top 30 BM25 passages with embeddings. The packaged v2 configuration is frozen to the benchmark.

Search defaults to v2, using an OpenAI-compatible embeddings service at http://localhost:1234/v1 with text-embedding-nomic-embed-text-v1.5 (as served by LM Studio in the benchmark). Override the service with --embed-url or $POCKETPEDIA_EMBED_URL, and the model with --embed-model:

pocketpedia search "Who directed Pixels" --retrieval v2 \
  --embed-url http://localhost:1234/v1 \
  --embed-model text-embedding-nomic-embed-text-v1.5

Vectors are cached in embcache.sqlite in the data directory. Changing the endpoint or model uses distinct cache keys. V2 sends questions and candidate Wikipedia passages to that endpoint; use a local service to keep processing offline. Changing models can change results, and the frozen prefixes are for the Nomic model. If embeddings are unavailable, search warns once on stderr and uses v1 for the rest of that command or MCP session. doctor probes the embedding endpoint without writing a cache; an unavailable optional endpoint does not make a healthy index fail its check.

MCP

The stdlib server uses newline-delimited JSON-RPC over stdio, protocol version 2025-06-18. Diagnostics go to stderr. It provides:

• wikipedia_search(query, k=5): titled passages with v2 and the warned v1 fallback.

• wikipedia_read(title, max_chars=20000, offset=0): full-text paging, with total_chars and next_offset. Offsets count characters; null means the end.

• wikipedia_lookup(title): existence, canonical title and lead, case-insensitively.

Claude Code setup (activate the environment containing pocketpedia first):

claude mcp add pocketpedia -- pocketpedia mcp --data-dir "$HOME/.pocketpedia"

For a generic MCP client that accepts mcpServers configuration:

{
  "mcpServers": {
    "pocketpedia": {
      "command": "pocketpedia",
      "args": ["mcp", "--data-dir", "~/.pocketpedia"]
    }
  }
}

The client must be able to find the installed pocketpedia executable. If it does not inherit your shell's PATH, set command to the executable in your virtual environment. Tool failures return isError: true and leave the server running.

Limits and attribution

This is English Wikipedia from November 2023. It cannot provide current facts, and retrieval can select the wrong entity or miss information outside the lead or passage window. read accesses the full stored article. There is no generative answer model in Pocketpedia; your client interprets the passages. V2 needs an embedding endpoint on a cold cache. V1 works entirely offline.

Code is MIT licensed, copyright Jon Hammant. Wikipedia text is downloaded by the user and is licensed separately under CC BY-SA 4.0, with some text also available under the GFDL; see Wikipedia's copyright notice. When quoting or redistributing text, credit Wikipedia and its contributors, link the source article and licence, indicate changes, and comply with share-alike requirements. Returned canonical titles identify the source: for example, Pixels (2010 film) corresponds to its Wikipedia article. The live article may differ from the 2023 snapshot. Preserve provenance when building answers from retrieved passages.

Development

pip install -e '.[build]'
python -m unittest discover -s tests
ruff check src tests
ruff format --check src tests

Tests build twenty invented articles in temporary directories and need no network or LM Studio. The subprocess test exercises the installed MCP command. An additional test compares 20 development questions with stored boosted passages if bench/data contains the existing index and PopQA files; it skips otherwise. The parquet build test runs when the build extra is installed. See bench/README.md for benchmark reproduction.