filings-search

Hybrid (BM25 + kNN) agentic retrieval over SEC 10-K filings, on OpenSearch, with a Claude tool-use agent that plans its own searches and answers with per-claim citations and a hard numeric-grounding check. FastAPI surface + TypeScript MCP tool server. Built as the OpenSearch successor to my Qdrant pipeline; the cite-or-abstain grounding is ported from my 12-domain provenance wrapper.

EDGAR ──► connector ──► section splitter ──► chunker ──► embeddings ──► OpenSearch (BM25 + HNSW kNN)
(SEC API)  (ticker→CIK,   (Item 1/1A/1C/7/8…)  (450 tok,    (nomic-embed-text            │
            10-K list,     order-constrained,    60 overlap)  local via Ollama, or        │
            iXBRL strip)   x-ref filtered)                     OpenAI)                     ▼
                                                                          hybrid search (RRF) + filters
                                                                                       │
                                                       ┌───────────────────────────────┼─────────────────────┐
                                                       ▼                               ▼                     ▼
                                              FastAPI /search /ask           TS MCP server            eval harness
                                              /chunk /companies         (search_filings, ask_filings)  (P@k/MRR, LLM-judge)
                                                       ▲
                                              Claude agent (tool loop):
                                              resolve_company → search_filings (item/ticker/FY filters, hybrid|bm25|knn)
                                              → expand_chunk → answer with [c:chunk_id] cites → numeric grounding check

What's in it

Run it

docker compose up -d                       # OpenSearch 2.19 (knn + neural plugins), :9200
ollama pull nomic-embed-text               # local embeddings
python3.12 -m venv .venv && ./.venv/bin/pip install -r requirements.txt
./.venv/bin/python ingest.py --recreate AAPL MSFT NVDA JPM XOM WMT TSLA JNJ   # ~2 min, 8 filings, ~2.1k chunks
./run_api.sh                               # FastAPI on :8801
./.venv/bin/python -m filings_search.agent "What does NVIDIA disclose about export controls to China?"
./.venv/bin/python eval/run_retrieval_eval.py
./.venv/bin/python eval/run_grounding_eval.py

MCP (Claude Desktop / Claude Code / Cursor):

{"mcpServers": {"filings-search": {"command": "node", "args": ["/ABS/PATH/filings-search/mcp/dist/server.js"],
                                    "env": {"FILINGS_API_URL": "http://127.0.0.1:8801"}}}}

Config via env: FS_OPENSEARCH_URL, FS_INDEX, FS_EMBED_BACKEND=ollama|openai, FS_AGENT_MODEL, FS_JUDGE_MODEL, SEC_USER_AGENT. ANTHROPIC_API_KEY (or ~/.env) for the agent/judge.

Results (2026-08-18, 8 filings / 2,101 chunks, 28 labeled queries)

Retrieval — see eval/retrieval_results.json:

Agent grounding — 10 analyst questions, claude-opus-5 agent + claude-opus-5 judge (see eval/grounding_results.json):

The one partially_grounded (Walmart tariffs, 0.87) was the agent summarizing a mitigation that the cited chunk states more narrowly — the judge caught it; that is what the judge is for.

Design notes / honest limitations

• Agentic ≠ fixed RAG. The model chooses company, Item, mode, and how many rounds; a fixed top-k pipeline gets no second chance. The tool descriptions carry the "when to use" guidance (item map, bm25-for-figures, widen-if-empty).

• Grounding is strict on purpose. A derived rounding ("$99,779M" → "~$99.8B") is flagged as ungrounded; analysts want the figure as filed. Loosen with a tolerance if you disagree.

• 10-K structure quirks are real, not parser bugs: JPM and XOM are "wrapper" 10-Ks whose MD&A/financials sit in a back-of-book Financial Section (labeled under the last Item); NVIDIA files statements under Item 15. Eval labels for those are ticker-only. A follow-up is F-page detection (Consolidated Statements of … headings) to relabel as Item 8.

• RRF is client-side — transparent and easy to reason about; OpenSearch's hybrid query

  • normalization pipeline is the in-cluster alternative. No cross-encoder re-ranker yet.

• Single filing per company in this run; --filings N pulls prior years (fiscal_year filter already in the mapping and tools).

• No auth, no rate limits, single-node OpenSearch — this is a working vertical, not a deployment.