seartch

A self-hosted search API that runs on a single Cloudflare Worker:

• /search: metasearch, a TypeScript port of SearXNG's core. It queries many engines in parallel, merges duplicates and ranks results with SearXNG's scoring.

• /fetch: fetches pages and returns their main content as clean Markdown, flagging paywalled or teaser articles. It can optionally fall back to the Wayback Machine for blocked or dead pages.

• /mcp: the same two capabilities as MCP tools (web_search, web_fetch), so Claude Code, Cursor, Zed and other MCP clients can use it directly.

It needs no containers, no paid search APIs and no API keys for any engine, and it runs on the Workers Paid plan ($5/month).

Features

• 23 ported SearXNG engines, plus engines for Indian financial news and stock research:

  • Exchange filings: NSE corporate announcements and upcoming board meetings, the same day's BSE announcements, and the latest BSE filings with one-line summaries from screener.in (which also covers BSE-only companies). Vague snippets like "General Updates" are replaced with the subject line read from the filing PDF, and filings with nothing to say are dropped. This answers "did X publish results?" from the source.

  • Index data: level, day change, 1-week/1-month/1-year price returns, 52-week range, P/E, P/B and dividend yield for any NSE index (NIFTY 50, Bank Nifty, sector indices, India VIX), shown as a card above the news.

  • Publishers: site search for Economic Times, Financial Express and Business Standard; Yandex restricted to nine financial publishers; and 26 live publisher RSS feeds (ET, Mint, Business Standard, CNBC TV18, BusinessLine, NDTV Profit, Business Today).

  • Regulators: RBI press releases and notifications, SEBI, and PIB. These run only when the query is about a regulator or regulation, and routine liquidity notices are skipped.

  • Google News (with link decoding) and Bing News, available by name.

• News mode (categories: ["news"] or ["filings"]):

  • Company resolution: maps the company a query names to every name it goes by (legal name, NSE ticker, BSE code, nicknames like RIL or HUL, the user's own wording). This needs an optional fuzzy ticker-search service; see Company resolution.

  • Relevance: drops stories that mention none of those names (no TCS stories in a Tata Elxsi search), listing, topic and quote pages, social posts, and gambling or crypto spam. Stories that only mention the company in passing rank lower.

  • Topics: in "Infosys deal wins", "deal wins" is the topic. Stories must name the company, and those that also match the topic rank first.

  • Matching: subject words must match; reporting jargon only ranks. "Q2", "Q2FY27", "September quarter" and "period ended September 30" count as the same thing.

  • Real publish dates: read from article URLs (Mint, Hindustan Times, Business Standard, dated paths) or page metadata, never from search-engine crawl dates.

  • Recency: relevance decays with age, sort: "date" gives newest first, and timeRange accepts quarter.

• SearXNG behaviour: URL normalisation and duplicate merging, weight × hits / position scoring, !bang shortcuts (!gh tokio, !science attention), categories, time ranges and paging.

• Engine suspension with backoff: an engine that serves a captcha, a 403 or a 429 is switched off for a while instead of being hit again. State is kept in memory per isolate, as in SearXNG.

• Per-engine result caching through the Cache API.

• Page fetch:

  • Readability extraction, then Markdown conversion.

  • Charset detection (non-UTF-8 pages decode correctly).

  • PDF and Office files converted with Workers AI toMarkdown, with image conversion disabled so it stays free. Scanned pages are counted and flagged; exchange results statements are often scanned past the cover letter.

  • Paywall detection from schema.org isAccessibleForFree plus barrier wording and teaser length.

  • Optional Wayback Machine fallback.

• Bearer-token auth, compared in constant time.

Related MCP server: WebVector MCP Server

Engines

What works depends heavily on where requests come from. Many engines block data-center IPs, including Cloudflare's. The defaults in src/searx/engines/index.ts are the engines that answered from a Cloudflare Worker in testing. Every engine can still be selected by name or !bang.

Results are best-effort. A site changing its HTML, or blocking an IP range, breaks or suspends that engine until its parser is updated.

Quick start

npm install --legacy-peer-deps
cp .dev.vars.example .dev.vars        # set AUTH_TOKEN
npx wrangler dev                      # http://localhost:8787

curl -X POST localhost:8787/search -H "Authorization: Bearer change-me" \
  -d '{"queries":["rust async runtime"],"limit":5}'

Deploy

npx wrangler deploy
openssl rand -hex 32 | npx wrangler secret put AUTH_TOKEN

Company resolution

News relevance and exchange filings get much better when the engine knows which company a query is about. That needs a fuzzy ticker-search service that answers:

POST /api/stock/fuzzy  {"query": "tata elxsi", "limit": 5}
→ {"data": [{"nseTicker": "TATAELXSI", "bseTicker": "500408", "companyName": "Tata Elxsi Ltd.",
             "isinNumber": "INE670A01012", "marketCap": 21350.3, "matchType": "exact_ticker", "confidence": 1}]}

Point seartch at it with either:

• A service binding named TICKER_SEARCH, for a Worker on the same Cloudflare account. Workers can't fetch() other Workers on the same account (error 1042), so this is the way for those.

• A secret TICKER_SEARCH_URL: npx wrangler secret put TICKER_SEARCH_URL.

Keep deployment-specific bindings out of the public config. For example, copy wrangler.jsonc to the gitignored wrangler.private.jsonc, add "services": [{ "binding": "TICKER_SEARCH", "service": "<your-worker>" }], and deploy with npx wrangler deploy -c wrangler.private.jsonc.

Without the service, nothing is resolved. News relevance then falls back to the query's subject words, and exchange filings return nothing.

Bindings and cost

wrangler.jsonc binds only Workers AI, and only for PDF and Office conversion. If you don't want it, remove the ai binding; those files then return an "unsupported" error. The fetch code can also use a Browser Run (BROWSER) binding as a fallback for JavaScript-heavy pages, but none is configured, because Browser Run is billed separately.

API

All endpoints except GET / need Authorization: Bearer <AUTH_TOKEN>.

POST /search

{
  "queries": ["tata elxsi q2 results", "!gh htmlrewriter"],
  "limit": 10,
  "categories": ["news", "general"],
  "engines": ["duckduckgo"],
  "timeRange": "week",
  "sort": "relevance",
  "verifyDates": true,
  "language": "all",
  "pageno": 1,
  "safesearch": 0,
  "includeContent": false,
  "archiveFallback": false
}

Up to 5 queries run per call. engines overrides categories. timeRange is day, week, month, quarter or year; sort is relevance or date. With includeContent: true, each result also carries its page as Markdown.

In news mode, each query's response also carries:

• companies: what the query resolved to, e.g. "Tata Elxsi Ltd. (NSE: TATAELXSI, BSE: 500408)".

• dropped: how many results were filtered as off-topic, non-article, spam or out of range.

verifyDates: false skips reading page metadata to date undated results.

POST /fetch

{ "urls": ["https://example.com/article"], "render": "never", "archiveFallback": true }

Each result has markdown, via (fetch / convert / wayback), and when relevant paywalled, partial, paywallSignals and archivedAt, or an error.

Others

• GET /engines: engine list with each engine's current suspension state.

• POST /engines/test: runs engines one by one and reports what each returned. Useful for checking which engines work from your deployment.

• POST /engines/reset: clears suspensions.

MCP

claude mcp add -s user --transport http seartch https://<your-worker>/mcp \
  --header "Authorization: Bearer <AUTH_TOKEN>"

Tools:

• web_search: queries, limit, categories, engines, timeRange, language, includeContent, maxContentChars.

• web_fetch: urls, render, archiveFallback, maxChars.

For news, have the client pass categories: ["news"] and a timeRange.

Development

npm test                 # unit + fixture tests in the Workers runtime (vitest + @cloudflare/vitest-plugin)
npm run typecheck
LIVE_ENGINES="duckduckgo,github" LIVE_QUERY="tokio" npx vitest run -c vitest.live.config.ts   # live engine check

Each engine lives in src/searx/engines/<name>.ts and implements request(params) and response(resp). Engines that don't map to a single HTTP request implement search(params, ctx) instead. Tests run each engine's parser against a saved response in test/fixtures/. The article pages used by the paywall tests are synthetic.

Please respect each site's terms of service and rate limits. Engines that scrape HTML are fragile by nature, and the suspension logic exists so blocked engines back off instead of hammering.

License

AGPL-3.0-or-later. This project ports engine and ranking code from SearXNG (© SearXNG contributors, AGPL-3.0). Under the AGPL, if you run a modified version as a network service, you must offer its source to its users.