<div align="center">

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kiycoh/silica-core/main/assets/banner.svg" />
  <img src="https://raw.githubusercontent.com/kiycoh/silica-core/main/assets/banner-light.svg" alt="Silica Core" width="590" />
</picture>

**Lightweight, local evidence retrieval tools for code and research**

Silica locates the source, symbol, page, or passage you and your agents need. Maximum signal, minimum machinery.

<p align="center">
  <a href="https://pypi.org/project/silica-core/"><img src="https://img.shields.io/pypi/v/silica-core?style=flat&labelColor=000000&color=000000" alt="PyPI" /></a>
  <a href="https://github.com/kiycoh/silica-core/blob/main/LICENSE"><img src="https://img.shields.io/github/license/kiycoh/silica-core?style=flat&labelColor=000000&color=000000" alt="License" /></a>
</p>

</div>

<!-- The MCP registry proves ownership of the PyPI package by finding this
     string in the description PyPI renders, which is this file. -->
<!-- mcp-name: io.github.kiycoh/silica-core -->

Silica indexes the markdown, code, PDFs and office files under one root and
serves them to [Claude Code, Cursor, Hermes, Codex, OpenCode and every other popular harness](#harnesses), as MCP tools or as shell commands that print the same
JSON. A hit is a path, a section, a line, a window of text and the numbers
to judge it by. The harness owns the loop.

<p align="center">
  <img src="https://raw.githubusercontent.com/kiycoh/silica-core/main/assets/quickstart.gif" alt="the quickstart recorded end to end: uv tool install, silica-core init reporting nine indexed documents, silica-core setup claude registering the MCP server, then Claude Code answering a question about the LSM compaction design space by calling silica-core and citing the PDF with its page and its line" width="900" />
</p>

Three commands, then a question asked the way you would ask any other. The
harness calls `silica_search`, and the answer carries the file, the page and
the line it came from.

## Install

In Claude Code, install [the plugin](#how-the-claude-code-plugin-works)
instead: it brings its own server, dense leg included.

```bash
uv tool install 'silica-core[mcp]'         # BM25 over documents and their sections (faster, lighter)
uv tool install 'silica-core[mcp,dense]'   # in addition the dense leg: numpy, a static model (still fast, more precise)
```

The second line adds the dense leg: `numpy` and a static model2vec model, no
torch and no GPU. It stays inert until the model is named and the sections
are embedded — the last stanza of the Quickstart. Take it when the questions
are paraphrases that share no words with the text; an exact term or an
identifier is answered by the lexical leg either way, and only that leg
reports `terms_absent`.

`pipx` works the same. The package and the command are `silica-core`, the
tools are `silica_*`.

Upgrading from 0.9 or older: the `silica` command is `silica-core` since
0.10.0 (`silica` belongs to silica-harness). A client registered by an older
`silica setup` still launches `silica mcp`, which stops starting once uvx
fetches 0.10.0. Run `silica-core setup <client>` once and it rewrites that
entry.

## Quickstart

In any folder of markdown, code, PDFs or office files:

```bash
silica-core init                              # adopt the folder: ignore file, vault.yaml, first index; the MCP server serves adopted folders only
silica-core search "leveled compaction" -k 5  # the best located passages
silica-core setup claude                      # register the MCP server, write the guidance block into ~/.claude/CLAUDE.md

# optional, with the [dense] extra: the dense leg, a static model, nothing leaves the machine
export SILICA_EMBEDDING_MODEL=model2vec/minishlab/potion-retrieval-32M
silica-core index --embed
```

<p align="center">
  <img src="https://raw.githubusercontent.com/kiycoh/silica-core/main/assets/search-hit.png" alt="silica-core search &quot;leveled compaction&quot; over nine LSM papers: the hit carries the file, p. 5 and the passage; beside it the PDF is open on that page with the same passage highlighted" width="900" />
</p>

Nine arXiv papers, indexed in 2.1 s. The hit names the file, the page and
the passage; the page beside it is the check.

## Tools

| Tool | Shell | Returns |
|---|---|---|
| `silica_files` | `silica-core files` | the inventory and what the index did with each file: `indexed`, `changed`, `excluded`, `failed`, `unconverted` |
| `silica_search` | `silica-core search` | ranked passages: path, section, line, BM25, matched terms, `coverage`, and the query terms absent from the corpus |
| `silica_read` | `silica-core read` | a slice by lines or by heading (a page, in a PDF), the outline, and a `version` to carry forward |
| `silica_code_pack` | `silica-core code-pack` | an AST context pack for one source file inside a character budget |
| `silica_write_note` | `silica-core write-note` | one atomic write, linted for structure and unresolved wikilinks |

In a source tree every function, method, class and constant is its own unit:
a hit's `section` is the symbol, `span` its lines, and
`silica_read(path, section=…)` serves the body. For a symbol whose name is
known, grep wins; for a question that names none, the search comes first:
the tool description and the block `silica-core setup claude` writes say so. The
contract, the reply shapes and the acceptance checks are in
[TOOLS.md](TOOLS.md). Nothing needs an API key or a network.

## How search says no

A ranked list always has a top, even when the corpus does not answer. Three
fields say how much the result is worth:

- `coverage`: the share of the query's idf mass the hit's matched terms carry. Near 1, every rare term matched; near 0, only common words did.
- `terms_absent`: query terms that occur nowhere in the corpus.
- `matched_terms`: the words this hit actually contains.

<p align="center">
  <img src="https://raw.githubusercontent.com/kiycoh/silica-core/main/assets/search-says-no.png" alt="silica-core search &quot;raft consensus log replication&quot; over the same nine papers: terms_absent lists raft and consensus, coverage falls to 0.19, and the top hit is a passage about data replication rather than Raft" width="900" />
</p>

`raft consensus log replication` over the same nine papers: `raft` and
`consensus` occur in none of them, `coverage` falls to 0.19, and the top hit
is about data replication. On 254 papers the top hit of an answered question
carries 0.69 to 1.00; a question the corpus does not cover, 0.44. Silica
exposes the signals; the harness decides whether to stop, read or rephrase.

## Benchmarks

nDCG@10 on BEIR SciFact · NFCorpus, the same documents and queries for every
arm. BEIR's published BM25 baselines are 0.665 · 0.325. Silica's lexical index
needs no model; the others serve lexical search from an index that also holds
embeddings.

| Mode | Silica | zvec-grep 0.2.2 | ck 0.7.11 |
|---|---:|---:|---:|
| Lexical | **0.662 · 0.311** | 0.649 · 0.297 | 0.630 · 0.289 |
| Hybrid, same `potion-retrieval-32M` embedder | **0.675 · 0.328** | 0.672 · 0.330 | |

Code, on the twenty SWE-QA questions zvec-grep publishes for its own
benchmark, same embedder, k = 10, scored on the files and symbols the
reference answer rests on.

| Arm | file hit@5 · @10 | file MRR | symbol hit@10 | symbol recall | chars returned |
|---|---:|---:|---:|---:|---:|
| **Silica**, hybrid | **0.85 · 0.90** | **0.68** | **0.75** | **0.24** | 8,266 |
| Silica, vectors | 0.80 · 0.85 | 0.67 | 0.65 | 0.21 | **6,225** |
| Silica, lexical | 0.65 · 0.75 | 0.47 | 0.45 | 0.14 | 8,194 |
| zvec-grep 0.2.2, hybrid | 0.65 · 0.75 | 0.54 | 0.55 | 0.17 | 7,326 |
| zvec-grep 0.2.2, vector | 0.60 · 0.80 | 0.61 | 0.55 | 0.18 | 6,821 |
| zvec-grep 0.2.2, FTS | 0.45 · 0.60 | 0.36 | 0.45 | 0.11 | 6,690 |

On BEIR the two hybrids tie at the 95% interval: the same vectors rank the
same, with no daemon and no vector store. On code, Silica's hybrid file MRR
is +0.135 over zvec-grep's hybrid (95% interval +0.01 to +0.27), paired per
question. The fusion also gains +0.21 MRR and +0.30 symbol hit over Silica's
lexical arm; no reranker or graph expansion is involved.

On this measured scope, Silica is a compact, local, SOTA-competitive
retriever: it matches zvec-grep on BEIR and leads the paired SWE-QA
code-localization replay with the same embedder.

Retrieval matters only if the agent does less work without losing the answer.
These are separate experiments and are not pooled:

| Workload and arm | Runs | Quality | Search used | Turns | Tool calls | Seconds | Warm cost |
|---|---:|---:|---:|---:|---:|---:|---:|
| Repository, search-first contract | 20 | Judge 59.7 | 17/20 | **4.7** | — | **24** | $0.197 |
| Repository, same plugin without contract | 20 | Judge 50.6 | 0/20 | 6.5 | — | 28 | **$0.180** |
| Documents, resident Silica tools | 12 tasks | 12/12 correct | 12/12 | **3.9** | **2.9** | — | **$0.20** |
| Documents, no plugin | 12 tasks | 12/12 correct | — | 5.0 | 4.0 | — | $0.22 |

The repository result is one repetition: turns improve by 1.75 (95% interval
0.55 to 3.05 fewer), while Judge and cost remain inconclusive. The document
rows belong to a 144-run study over twelve questions and a 5.5M-token corpus.
They establish less work on that workload, not a universal agent claim.

Corpora, intervals, per-task exceptions and reproduction commands are in
[benchmarks](public/benchmarks.md).

## Harnesses

`silica-core setup <client>` writes the registration into the client's own config
and backs up what was there; for `claude` it also puts a guidance block, when
to search before grep, into `~/.claude/CLAUDE.md`. `silica-core setup --list`
names the clients: `claude`, `codex`, `cursor`, `windsurf`, `zed`, `cline`,
`roo`, `continue`, `goose`, `opencode`, `openhands`, `gemini`, `antigravity`, `dsh`,
`hermes`, `openclaw`, `agent-zero`, `claude-desktop`, `lmstudio`,
`anythingllm` and `librechat`; `shell`, `python` and `generic` print recipes
for anything else. The server serves the folder the client opens in;
`--vault DIR` or `SILICA_VAULT` fixes the root.

Every written block, and the Claude Code plugin, run `silica-core mcp --retrieval
local-hybrid`: `potion-retrieval-32M` in the server process, index and
vectors built in the background at start, the search lexical and `dense:
warming` until they land. Nothing leaves the machine; the one download is
the model, once. `npx skills add kiycoh/silica-core` installs the skills (the one
that tells an agent when to reach for the tools, and five that explain,
summarize, compare, schematize and diagram from the indexed documents), and
nothing else. Shell recipes,
Docker and the REPL are in [public/harnesses.md](public/harnesses.md).

## How the Claude Code plugin works

In Claude Code the plugin is all you need: it brings its own server and the
skills. The [Install](#install) section above and `silica-core setup claude`
are for other clients; with both, two servers serve the same folder.

### Install

It needs [uv](https://docs.astral.sh/uv/) on `PATH`, which starts the server.

```bash
claude plugin install silica-core --marketplace kiycoh/silica-core
```

That one step works on Claude Code 2.1.275 or later, and
`/plugin install silica-core --marketplace kiycoh/silica-core` does the same
inside a session. On an older client, add the marketplace first:
`claude plugin marketplace add kiycoh/silica-core`, then
`claude plugin install silica-core@silica-core`.

### First run

1. Start Claude Code in the folder you want to search.
2. Wait for the first start. It is slower than the next ones: uv installs the
   packages and the server downloads its model, once.
3. Press **Index** above the prompt, or type `/silica-init`. Nothing is
   indexed until you do.
4. Ask a question about the folder. Claude searches it with `silica_search`.
   Until the vectors are built the search is lexical, and its replies say
   `dense: warming`.

### Update

Refresh the marketplace, then the plugin. The new version loads in the next
session.

```bash
claude plugin marketplace update silica-core
claude plugin update silica-core@silica-core
```

### What it runs, reads and downloads

The plugin is an MCP server plus a small mod in `hooks/`. Neither
one sends your files, prompts or conversation anywhere.

- **The server** starts with `uv run --frozen --project <plugin folder>`.
  The plugin folder is
  `~/.claude/plugins/cache/silica-core/silica-core/<version>`, with the
  version `claude plugin list` shows. The first time, uv installs the
  packages pinned in `uv.lock` from PyPI (and a Python build if the machine
  has no Python 3.11 or newer), and the server downloads the
  `potion-retrieval-32M` model from Hugging Face. Nothing else is
  downloaded.
- **Indexing.** In a folder that isn't indexed yet, the mod shows an
  **Index** button above the prompt; `/silica-init` does the same. Either one
  runs `uv run --frozen --project <plugin folder> --extra mcp --extra dense
  silica-core init` in that folder, which writes `vault.yaml` there (plus
  `.silicaignore` in a code repository) and builds the index under
  `~/.silica`. **Not here** hides the button for that folder. Nothing is
  indexed until you ask.
- **Search hints.** In an indexed folder, when you ask a question that names
  no file or symbol, the mod adds one line asking Claude to try
  `silica_search` before grep. It also adds the search guidance to the system
  prompt, unless your `CLAUDE.md` already has it. The mod reads three
  environment variables, `CLAUDE_CONFIG_DIR`, `HOME` and `USERPROFILE`, only
  to find that `CLAUDE.md` and to skip your home folder.
- **Docs upkeep.** In an indexed folder with a `docs/` folder, the mod helps
  Claude keep the docs in step with the code. It asks; Claude decides, and
  no write is ever refused.
  - After a write worth recording (a new file, a manifest, 30 changed lines
    in a turn, or a file an ADR lists), Claude gets one reminder: bring
    `docs/` up to date, or say there is nothing to record.
  - When a turn that wrote under `docs/` ends, the mod checks those files
    for links and code paths that lead nowhere, two ADRs with one number,
    and an accepted ADR whose Decision changed. Claude fixes what it agrees
    is a defect or says why it is not; the check asks at most twice a turn.
  - At the start of an interactive session it reads `docs/adr/` and
    `docs/specs/` and runs `git`, then shows above the prompt how many
    records name what is not there and how many have files that changed
    since. `/silica-docs` lists them.
  - A record whose frontmatter has `implementation:` (`implemented`,
    `partial`, `pending`, `draft`, `dropped` or `n/a`) is held to it:
    `implemented` and `partial` name a commit or a path in `implemented_in`,
    `pending` and `draft` name none, `partial` and `dropped` carry an
    `implementation_note`.

### On an older Claude Code

The **Index** button, `/silica-init`, the search hints and the docs upkeep
come from the mod, which needs Claude Code 2.1.293 or later. An older client
still runs the server and the skills. Index a folder from a terminal opened
in it instead:

```bash
uv run --frozen --project <plugin folder> --extra mcp --extra dense silica-core init
```

### When it does not work

- **The silica tools are missing, or Claude Code says it could not connect to
  `plugin:silica-core:silica-core`.** The server did not start; `/mcp` shows
  its state. The usual causes are uv missing from `PATH` and a first install
  that failed. Start the server by hand in the folder to read the error:

  ```bash
  uv run --frozen --project <plugin folder> --extra mcp --extra dense silica-core mcp --retrieval local-hybrid
  ```

  When all is well it prints `serving 5 tools on stdio, waiting for a client`;
  Ctrl+C ends it.
- **No Index button.** It never shows in your home folder or in `/`, which
  are too large to index whole. After **Not here** it stays hidden in that
  folder, and `/silica-init` still indexes it. On a client older than
  2.1.293, see above.
- **What this machine can read.** `silica-core doctor` checks the install
  and the folder:

  ```bash
  uv run --frozen --project <plugin folder> --extra mcp --extra dense silica-core doctor
  ```

### Uninstall

```bash
claude plugin uninstall silica-core@silica-core
```

The plugin leaves two things behind:

- `vault.yaml`, and `.silicaignore` in a code repository, in each folder you
  indexed;
- the indexes in `~/.silica/index/`, one folder each.

Delete them by hand. An index deleted while the plugin is still installed is
rebuilt the next time its folder is searched.

## Notes

- **What it reads:** markdown, `.txt`, `.rst` and PDFs with a text layer directly, one PDF page per section; DOCX, EPUB, FB2, RTF, XLS and ODF converted with no extra; scanned PDFs, images, PPTX and XLSX through `silica-core import` with [MinerU](https://github.com/opendatalab/MinerU); audio and video with `ffmpeg` plus `SILICA_STT_BASE_URL`; CSV readable by line, excluded from search. In a source tree the code lane adds source files and their `json`, `yaml`, `toml`, `cfg` and `ini`. `silica-core doctor` says which lanes this machine has.
- **The dense leg:** section embeddings that catch a paraphrase sharing no rare word with the answer. `uv tool install` stays lexical until `silica-core index --embed`, with the `[dense]` extra's local model or any OpenAI-compatible `/v1/embeddings` endpoint. Text leaves the machine only for a remote endpoint, and only after `silica-core index --embed --allow-remote` grants that host once. Variables and reply states in [TOOLS.md](TOOLS.md), the ollama recipe in [public/harnesses.md](public/harnesses.md).
- **More surfaces:** `silica-core mcp --extended` adds the wikilink tools; `silica-core connect` (extra `[connect]`) hosts the bridge the Obsidian plugin dials into, so writes land through the vault API while the app is open; `silica-core repl` runs a small reference agent over the same tools, the one surface that needs a model (`SILICA_MODEL`).
- **Not in the core:** no memory lane, prompt injection, summaries or undo journal. Undo is git.

## License

MIT. See [LICENSE](LICENSE).