NB Paper Search

English | 简体中文

NB Paper Search is a standalone research-discovery CLI. One core, one contract, four surfaces: a human terminal, an MCP server, a batch runner, and an installable Agent Skill. Search sources and PDF/extraction handlers are not baked into that core. They arrive later as zip packages from subscription registries, so a new database or downloader does not require a fork.

It belongs to the nb-* product family. The npm package is nb-paper-search (private, unpublished). The command you type is still paper-search, the same way GitHub CLI is gh. Config home, environment variables, the MCP server name, the Skill slug, and provider version-gate wording keep their existing paper-search / paper-search-cli identities so current installs, Paperflow, and provider packages keep working.

Use it when you need literature or patent discovery that you can replay, a citation graph you can bound and resume, assessment that stays checksum-bound, and artifact work that records who fetched what. Ordinary search hits are candidates, not a bibliography. Promotion into a workspace, evidence store, or Zotero is always an explicit selected-record action.

Current version is 0.5.0.

Command, not package name

After install, every example in this README starts with paper-search:

paper-search --version
paper-search paths
paper-search doctor

Do not type nb-paper-search at the shell. That is the repository and package name. The shim, the Skill launcher, and package.json bin all expose paper-search.

Unchanged live contracts:

Related MCP server: scholar-toolkit-mcp

First meaningful search

This is a source-linked install plus at least one search provider. It is not a five-minute npm i -g. You need Git, Node.js 20 or newer, and npm 10 or newer (the tested release is packageManager in package.json). Keep the clone; the runtime lives inside it.

git clone https://github.com/X-T-E-R/nb-paper-search.git
cd nb-paper-search
node scripts/install.mjs
node scripts/install.mjs --apply

The first command only prints a plan. --apply builds a verified runtime in the checkout, projects the Skill to ~/.agents/skills by default, and creates the paper-search shim under ~/.paper-search/bin unless you pass --bin-dir. The installer reports whether that directory is on PATH; it does not edit your environment.

paper-search --version
paper-search doctor

Core ships no academic source. Subscribe to the search registry, refresh it, and install a provider that can run without extra credentials:

paper-search registries add official-search \
  https://github.com/X-T-E-R/resource-search-providers/releases/download/providers-registry-latest/registry.json \
  --kind search
paper-search registries add official-search \
  https://github.com/X-T-E-R/resource-search-providers/releases/download/providers-registry-latest/registry.json \
  --kind search --apply
paper-search registries refresh official-search
paper-search providers install arxiv --apply

Then keep runs with the project instead of only the global home:

paper-search context init . --id my-review
paper-search search-plan --type academic --source arxiv
paper-search academic "retrieval augmented generation" --source arxiv

search-plan shows expansion and readiness without sending a query. The search writes one durable run under the nearest context runs.root (default for a standalone context: .paper-search/runs). That run is a record of the query, not an accepted bibliography entry. To keep a hit, select it later with resource-add.

Install from a retained checkout

Clone the official repository and leave that directory in place. Direct npm install -g is not the supported path.

git clone https://github.com/X-T-E-R/nb-paper-search.git
cd nb-paper-search
node scripts/install.mjs
node scripts/install.mjs --apply

Project the Skill into extra agent roots, or put the shim somewhere already on PATH:

node scripts/install.mjs \
  --target ~/.agents/skills \
  --target ~/.codex/skills \
  --bin-dir ~/.local/bin

node scripts/install.mjs \
  --target ~/.agents/skills \
  --target ~/.codex/skills \
  --bin-dir ~/.local/bin \
  --apply

Once the shim directory is on PATH:

paper-search --version
paper-search paths
paper-search self status
paper-search status --json

paper-search setup is the same plan-first installer, used later to inspect or repair the verified build, Skill projections, and shim. Add --apply only after reviewing the plan.

paper-search setup
paper-search setup --target ~/.agents/skills --target ~/.codex/skills --apply

Agents should enter through the projected Skill launcher, skills/paper-search-cli/scripts/paper-search.mjs. Both that launcher and the paper-search shim resolve back to the retained checkout. No runtime is copied into the Skill directory.

Manage the retained checkout

New installations default to user-managed: Paper Search never treats an installer-owned checkout as permission to update its source.

paper-search self mode
paper-search self mode user-managed
paper-search self mode user-managed --apply
paper-search self mode self-update
paper-search self update

self mode and self update are plan-first. A write occurs only when the plan passes its safety checks and you repeat the command with --apply. Self-update requires an installer-owned, clean, attached checkout with a matching official upstream and a fast-forward-only target. Local-only or diverged commits block the operation.

The production build seals its official origin to https://github.com/X-T-E-R/nb-paper-search.git on main. Configuration files, environment variables, and flags cannot add or override an origin. An SSH clone, fork, different branch, dirty checkout, or local-only commit remains user-managed; update it with ordinary Git commands and then run paper-search setup. An installer-owned clean HTTPS clone of that exact origin may opt into self-update after reviewing the mode plan.

Repository development

node dist/cli.js ... is for work inside the checkout, not installed use.

npm ci
npm run build
npm run check
node dist/cli.js status --json
node dist/cli.js tools --json

Configure

paper-search config path --all shows the conventional bundle. All conventional user state lives below ~/.paper-search/. Old %APPDATA%/paper-search, $XDG_CONFIG_HOME/paper-search, and ~/.config/paper-search locations are migration inputs, not live authorities.

• config.toml — user-owned, non-secret runtime settings. This file is the only required conventional file. Example: paper-search.example.toml.

• subscriptions.toml — trusted registry definitions, validated separately. Example: subscriptions.example.toml.

• credentials.toml — optional plaintext credentials with restricted filesystem access. It is not encrypted; prefer environment variables when plaintext-at-rest is unsuitable.

• external-search.toml — optional External Search v1 process authority. It is not part of the layered config merge. Example: external-search.example.toml.

Project settings come from the nearest ancestor paper-search.toml or .paper-search.toml. If both exist in the same directory they merge in that order and the CLI reports a compatibility warning. --config <path> selects an additional explicit file, or a directory containing config.toml.

Default local records are separated by purpose:

~/.paper-search/
  config.toml              config.d/
  subscriptions.toml       credentials.toml
  external-search.toml     adapters/
  providers/               cache/registries/
  cache/archives/          state/
  workspace/               runs/
  storage/artifacts/       storage/extractions/
  exports/                 bin/

Override future locations with workspace.root, storage.artifactRoot, storage.extractionRoot, storage.exportRoot, and runs.root. Existing legacy path fields keep their original workspace-relative meaning. The default runs.maxAgeDays = -1 disables age-based eligibility. Paper Search never prunes runs during another command. Durable history is private local plaintext until you run runs prune --apply. runs.recordByDefault = true records real discovery through the friendly CLI, canonical/MCP, and batch surfaces. A direct CLI command may pass --no-history; canonical/MCP callers may send recordHistory: false. Planning and dry-run stay write-free.

Plain discovery does not need a destination flag. From the invocation directory, Paper Search walks upward to the nearest project config. If that file declares a standalone or Paperflow context, the full run is written only to its runs.root; otherwise it goes to the effective global run root (~/.paper-search/runs by default). Context runs also get a private global locator so runs show <id> still works outside the project.

paper-search context init . --id my-review
paper-search context status
paper-search academic "retrieval augmented generation" --source arxiv

context is project/explicit configuration only. User config.toml cannot declare one, and global is reserved for the built-in fallback. A fresh Paperflow workspace already ships a root paper-search.toml that maps runs.root to search_runs, so no --paperflow or --save flag exists or is needed. --no-history creates neither a run nor a locator.

Effective values, lowest to highest:

1. built-in defaults

2. user config.toml

3. nearest ancestor project config

4. explicit --config

5. user credentials.toml for credential keys

6. PAPER_SEARCH_* environment variables

7. command-specific flags

Project and explicit config may override ordinary runtime values but are not promoted into trusted subscriptions. paper-search config explain <key> shows the winning value and origin. paper-search config validate checks conventional, project, and explicit files against their owning schemas.

Use dedicated credential commands instead of generic config set for installed academic, patent, or material providers:

printf '%s' "$WOS_API_KEY" | paper-search config credentials set platform.wos.apiKey --stdin
paper-search config credentials get platform.wos.apiKey
paper-search config credentials unset platform.wos.apiKey

Credential values are never accepted as positional arguments. get masks stored values. config import-env plans by default: non-secrets go to config.toml, secrets to credentials.toml, and secrets are masked in output.

paper-search config import-env ./.env
paper-search config import-env ./.env --apply

Entries already set in the current shell are skipped, so importing a file does not silently override the environment that currently wins.

Common environment overrides:

• PAPER_SEARCH_HOME

• PAPER_SEARCH_PROVIDERS_INSTALL_DIR

• PAPER_SEARCH_WORKSPACE_ROOT

• PAPER_SEARCH_STORAGE_ARTIFACT_ROOT

• PAPER_SEARCH_STORAGE_EXTRACTION_ROOT

• PAPER_SEARCH_STORAGE_EXPORT_ROOT

• PAPER_SEARCH_RUNS_ROOT

• PAPER_SEARCH_RUNS_MAX_AGE_DAYS

• PAPER_SEARCH_RUNS_RECORD_BY_DEFAULT

• PAPER_SEARCH_ZOTERO_ENABLED

• PAPER_SEARCH_ZOTERO_ENDPOINT

Search source selection

Source classification, request selection, and runtime readiness are independent. Web of Science can remain a multidisciplinary catalogue member with no credentials: general still selects it, the run reports it skipped, and it starts working after valid configuration without a taxonomy edit.

Academic search without a positive selector uses general; patent search uses patents. Built-in presets: general, computer-science, biomedicine, preprints, repositories, publishers, and patents. general is domain:multidisciplinary. DBLP is computer-science, not general.

CLI selectors are repeatable. Positive selectors form a canonical-id union; as soon as one is present, the implicit command default is not added. Exact source exclusions are final.

paper-search academic "retrieval augmented generation"
paper-search academic "graph neural networks" --preset general --preset computer-science
paper-search academic "single-cell transcriptomics" --category domain:biomedicine --source crossref
paper-search academic "foundation models" --preset general --exclude-source wos
paper-search academic "formal verification" --platform all
paper-search search-plan --type academic --preset general --category domain:computer-science

paper-search configure --json lists only pending provider setup. Target one provider with paper-search configure <id>. Non-interactive and JSON invocations never prompt; they return a credential-free command action. Absent platform.<id>.enabled means auto, true means explicitly enabled, and false disables both execution and setup reminders.

Academic searches accept --sort-by relevance|date|citations; patent searches accept --sort-by relevance|date. date and citations are descending. Sorting stays inside each provider group because citation counts and pagination are not comparable across sources. Invalid friendly CLI values fail immediately instead of falling back to relevance.

Literal --platform all selects every installed, valid, configured, enabled, non-view source for that command type. It does not mean general. Views are selected only by exact id or an explicit user preset.

Keep a small config.toml and split optional settings into config.d/*.toml (lexical order, same schema). Project files use matching names such as paper-search.toml plus paper-search.d/*.toml.

Legacy single-file configuration and flat provider directories are migration inputs. migrate plans config and provider-directory work together and applies only with --apply. An explicit --legacy-install-dir is required before a custom provider root may be moved. Migration never rewrites project configuration.

paper-search migrate
paper-search migrate --apply
paper-search migrate --legacy-install-dir /path/to/custom/providers --apply

Extend with providers

Search sources and material handlers are installable zip packages from two subscription registries. Core contains no source-specific PDF downloader and no network extractor.

• Search providers come from X-T-E-R/resource-search-providers (23 published providers in the compatibility matrix: acm, arxiv, biorxiv, core, crossref, dblp, europepmc, iacr, ieee, medrxiv, openaire, openalex, openreview, patentstar, pmc, pubmed, sciencedirect, scopus, semantic, springer, usenix, wos, zjusummon).

• Material providers come from X-T-E-R/material-providers (two published packages you can install today: unpaywall, mineru-extractor). The same repository also contains in-progress direct-url-downloader and local-pymupdf4llm packages that the live registry does not serve yet. See ADR-0002.

Subscribe, then install. Trust changes are plans until --apply. refresh validates metadata and writes a digest-addressed snapshot; it does not install providers. A package whose minCliVersion is newer than this CLI is blocked with requires paper-search-cli >= X.

paper-search registries add official-search \
  https://github.com/X-T-E-R/resource-search-providers/releases/download/providers-registry-latest/registry.json \
  --kind search --apply
paper-search registries add official-material \
  https://github.com/X-T-E-R/material-providers/releases/download/material-registry-latest/registry.json \
  --kind material --apply
paper-search registries list
paper-search registries show official-search
paper-search registries refresh official-search
paper-search providers available --json
paper-search providers install arxiv --apply
paper-search providers install unpaywall --from official-material --apply
paper-search providers list-installed --kind search
paper-search providers list-installed --kind material

If more than one active subscription publishes the same provider id, install requires --from <subscription-id>. Bound installs pin subscription identity, registry digest, archive digest, package identity, and the installed-state precondition. Applying a stale plan is rejected. Updates follow the source recorded in the installed receipt.

The catalogue describes endpoints and technical configuration. It does not decide jurisdiction, licence, entitlement, or authorization. You choose which sources to enable.

Search-provider bundles run in the CLI process. The compatibility vm wrapper is not a hostile-code sandbox. Registry id, version, SHA-256, minimum-version, and ZIP-path checks protect release identity, not against malicious provider code. Install only from registries or archives you trust.

GitHub repository shorthand expansion is a search-provider compatibility feature. Material registries require the exact registry.json URL.

Low-level inspection and unbound zip workflows remain on providers --kind search|material. material-providers is a compatibility alias for --kind material. sync-registry and install-zip do not authorize later subscription-bound updates. install-zip refuses to replace a bound provider unless you preview with --replace-bound and then --apply. uninstall is plan-first and retains the exact provider directory and receipt; restore with providers rollback <id> --kind <kind> --revision <sha256>.

Bound packages use kind-separated directories: search/<id> and material/<id>. Provider ids are globally unique across both kinds. Existing flat packages remain readable until paper-search migrate moves them.

Search, lookup, and workspace

paper-search academic "retrieval augmented generation"
paper-search patent "solid state battery" --platform patentstar --database CN --patent-type invention
paper-search patent-detail patentstar ANE123 --include legalStatus,claims,pdf
paper-search web "latest RAG evaluation benchmarks" --mode deep --freshness pm --max-results 5
paper-search lookup "10.1145/3366423.3380130"
paper-search resource-add --item-file ./search.json --index 0 --collection-path Research/Inbox --tags rag --json
paper-search resource-pdf <workspace-item-id> --url https://example.org/paper.pdf --filename paper.pdf --json
paper-search collection-list --flat --json
paper-search workspace-export --collection-path Research --include-children --out ./paper-search-export.bib --json
paper-search workspace-export --collection-path Research --store reports/research.bib --dry-run --json

lookup is the recommended step before resource-add when you already have a DOI, PMID, arXiv ID, ISBN, or URL. patent-detail is the recommended step before resource-add when claims, legal status, or PDF URLs matter. resource-pdf and pdf are compatibility entrypoints for existing workspace item ids. They use the same installed material-provider path as artifact download; core does not fetch the URL. Prefer artifact and extraction records for new material work.

web exists only when user-level external-search.toml grants External Search v1 process authority. Project and --config files cannot grant it. status never starts that process; doctor runs the protocol-defined no-network probe.

Downloads use material.downloadDisposition = "selected" by default: after bytes are committed, Paper Search creates or reuses a workspace item by DOI, source id, or URL and attaches the artifact. Set "materialized" to keep downloads standalone until an explicit resource-add or attachment. Extraction alone does not imply selection.

workspace-export --store <safe-relative-key> writes atomically below storage.exportRoot, rejects an existing target, and supports --dry-run. --out <path> remains an explicit caller-working-directory path. Omitting both writes to stdout.

Durable runs, citation, and assessment

Real academic, patent, lookup, optional web, canonical/MCP, and batch discovery calls are recorded by default. paper-search run <canonical-tool> is the explicit always-durable form for academic_search, patent_search, resource_lookup, patent_detail, and optional web_search.

paper-search runs list
paper-search runs show <run-id>
paper-search runs export <run-id> --out ./run.json
paper-search runs pin <run-id>
paper-search runs prune
paper-search runs prune --apply

Plan citation expansion before starting it. Keep traversal limits explicit. Resume an interrupted durable run by run id.

paper-search citation plan --doi 10.1145/3366423.3380130 --direction backward --depth 1 --max-nodes 50
paper-search citation run --doi 10.1145/3366423.3380130 --direction backward --depth 1 --max-nodes 50
paper-search citation status <run-id>
paper-search citation resume <run-id>

Assessment inspects checksum-bound observation snapshots, provenance, conflicts, and an optional explicit policy trace. Paper Search does not choose which papers you should accept.

paper-search assess plan --snapshot ./snapshot.json --sha256 <digest>
paper-search assess run --snapshot ./snapshot.json --sha256 <digest>
paper-search assess show <run-id>
paper-search assess list

Contracts for durable runs, citation, assessment, storage, material providers, and Zotero live in NB Paper Search workflows and storage.

Artifacts and extraction

Material workflows store auditable metadata under the configured workspace root. Bytes and extracted outputs use storage.artifactRoot and storage.extractionRoot.

artifact download and material ingest accept a DOI. An installed artifact_resolver (select with --resolver <id>) turns it into ordered candidate locations. Dry-run plans list load-resolver and run-resolver steps. Resolver failures are typed as no_resolver, no_candidates, or resolver_error.

paper-search artifact download 10.1038/nature12373 --resolver unpaywall --dry-run --json
paper-search artifact list --json
paper-search artifact show <artifact-id> --json
paper-search material ingest https://example.org/paper.pdf --dry-run --json
paper-search extract <artifactId-or-path> --provider local-pymupdf4llm --json
paper-search material status <workspace-item-id-or-artifact-id-or-extraction-id> --json

Published material packages:

• unpaywall — DOI resolver.

• direct-url-downloader — explicit HTTPS artifact URLs through the injected material HTTP runtime. Providers do not receive raw fetch.

• mineru-extractor — document-model extraction, including extensionless arxiv.org/pdf/... URLs. Local-only inputs are unsupported until the host implements MinerU's signed batch-upload flow.

• local-pymupdf4llm — explicit offline extraction of a managed PDF. Never chosen implicitly. Create its isolated Python 3.11 runtime, then name the provider on each extraction:

paper-search material setup-local-pymupdf4llm --python C:\Path\To\python.exe
paper-search material setup-local-pymupdf4llm --python C:\Path\To\python.exe --apply
paper-search extract <artifactId-or-pdfPath> --provider local-pymupdf4llm --policy local-offline-pdf --json

The runtime is fixed to pymupdf4llm 0.3.4, PyMuPDF 1.27.2.3, and tabulate 0.10.0 under the Paper Search home. Review the AGPL/commercial licence choice for that deployment. network: false; OCR is absent, so an OCR request fails as OCR_UNAVAILABLE instead of silently degraded text.

Optional institutional browser continuation is a user-mediated DOI fallback, disabled by default, and accepted only in ~/.paper-search/config.toml. artifact download <DOI> --institutional tries the normal resolver/downloader first. Canonical and MCP callers can create and inspect sanitized jobs but cannot continue them. Paper Search does not install Python packages or browser runtimes. See NB Paper Search workflows and storage.

Zotero

Zotero is optional. Configure a global selected-item policy or a workspace binding to project metadata and local files through Zotero MCP Neo without changing the authoritative local copy. Only conventional user configuration may enable the writer. General Zotero library work belongs in a dedicated Zotero skill; these commands cover Paper Search's mapped projection.

paper-search zotero status
paper-search zotero sync <itemId>
paper-search zotero sink <itemId>
paper-search zotero sink <itemId> --preview
paper-search zotero sink <itemId> --apply --ack <digest>

zotero sink plans by default. Writes require a current preview digest and --apply.

Paperflow boundary

Paperflow is the directory and research-workflow layer, not the search engine, and not a prerequisite. Fresh Paperflow workspaces generate a paper-search.toml whose runs.root matches their search_runs role. Paperflow then reads mounted run history:

paperflow search history
paperflow search show <run-id>
paperflow search candidates <run-id>

Paper Search does not parse paperflow.yaml. It never promotes search hits into a Paperflow bibliography or evidence store. That promotion remains a selected-record action on the Paperflow side. See Paperflow integration.

Batch, precise tools, and MCP

batch accepts CSV, JSONL, JSON, and YAML task files. Supported rows include search, lookup, workspace, artifact, extract, material ingest, citation_expand, and assessment_run. With --resume-from ./results.jsonl --out ./results.jsonl, rows append JSONL results and keep both row index and row id.

paper-search run academic_search \
  --json-args '{"query":"retrieval augmented generation","sources":["arxiv"],"maxResults":5}'
paper-search run resource_lookup --arg identifier=10.1145/3366423.3380130

mcp serve exposes the same config, provider runtimes, result envelopes, capability tags, and workspace sink as the CLI. The server identity is paper-search-cli-mcp.

• default HTTP endpoint: http://127.0.0.1:23121/mcp

• help: http://127.0.0.1:23121/mcp/help

• status: http://127.0.0.1:23121/mcp/status

• JSON-RPC methods: initialize, initialized, notifications/initialized, tools/list, tools/call, resources/list, prompts/list, ping

• --transport stdio for line-delimited JSON-RPC clients

The HTTP endpoints are a project-specific JSON-RPC bridge, not a complete MCP HTTP+SSE transport. Use --transport stdio for standards-based clients.

paper-search mcp serve
paper-search mcp serve --transport stdio

Every command and canonical tool belongs to one capability group. That map is the routing contract for the CLI, MCP tools, and companion Skill:

Machine-readable commands return a ResultEnvelope with ok, capability, tool, data, and optional planned, diagnostics, warnings, errors, and provenance. Dry-run output sets planned: true. Setup actions contain no credential values and point to paper-search configure <id>.

Limits, honesty, and compatibility

• Hits from academic, patent, web, and lookup are not accepted evidence and are not bibliography entries.

• Core does not download source-specific PDFs or extract over the network. Install material providers for that work.

• Search providers execute in-process. Treat the registry as a trust boundary.

• External web search is optional and user-home gated. Version 0.4 removed the built-in Tavily, Firecrawl, Exa, xAI, and MySearch adapters and removed web_research. See Migrating to 0.4.

• Institutional acquisition is local, interactive, and off by default.

• Self-update cannot be pointed at a fork by config.

Default tests are deterministic and offline. The suite passes on Linux and macOS. On a Windows host a small number of tests fail for host-environment reasons (an ACL path that needs native Windows whoami, and a flaky atomic-rename EPERM); those are host limits rather than product defects.

npm run test:unit
npm run test:contract
npm run test:integration
npm run test:compat:official
npm run test:smoke
npm run verify:release

Without PAPER_SEARCH_RUN_SMOKE=1, npm run test:smoke exits with a skipped summary. Live cases and required environment variables are in docs/testing.md.

Architecture and contributing

Layering, provider runtimes, MCP, and test boundaries: NB Paper Search architecture.

There is no separate contributing guide. Work from a retained checkout, keep contracts stable across the four surfaces, and run npm run check before proposing a change. Search and material packages live in their own repositories; this repo owns the CLI, docs, tests, companion Skill, and installer.

License

NB Paper Search is available under the MIT License.