Skip to main content
Glama
free-search-mcp-ts

free-search-mcp-ts

by lhswgzy

free-search-mcp-ts

Local-first web search, page fetching and document parsing for any MCP client — with no API key.

By lhswg (GitHub lhswgzy) · © 2025 · MIT licensed.

Origin. This repository is an independent, from-scratch TypeScript implementation. The idea, and the older and more mature Python project that came first, belong to sweetcornna: sweetcornna/free-search-mcp — MIT, published on PyPI as free-search-mcp. This is not a fork, none of its code was copied or read, and no parity is claimed. Full statement: AUTHORS.md.

free-search-mcp-ts is a Model Context Protocol server that gives Claude, GPT, Cursor, Codex, local Ollama front ends and any other MCP-capable client the ability to search the web, read pages and parse documents. It runs entirely on your machine, needs no account, and returns Markdown instead of JSON because Markdown costs a model roughly a third fewer tokens for the same information.

Not on npm yet. npx -y free-search-mcp-ts will install it once the package is published; until then use the Quick start, which builds it locally in four commands.

CI License: MIT Node Author


Why this exists

Asking a model to "look it up" usually means one of three things: pay for a search API, hand the model a browser it cannot drive, or paste URLs manually. None of them work well for a local model, and none of them keep your queries on your machine.

This server takes a different position:

  • No keys by default. DuckDuckGo, Mojeek and Google News are queried directly over their public endpoints. Nothing to sign up for, nothing to leak.

  • Several engines, not one. Every engine's result list is merged with Reciprocal Rank Fusion, so a page that three engines independently surface outranks a page only one engine found. The result set is stable in a way a single provider's is not.

  • It degrades instead of failing. Engines that are blocked, rate-limited or unreachable are detected, benched, and replaced by the next tier. The server keeps working on networks that filter some providers — one of the engines in the default fallback tier is there precisely because it stays reachable where others do not.

  • Everything stays local. Queries go to the search engine; the pages you read are stored in a local SQLite index on your disk, and nothing else is uploaded anywhere. There is no telemetry.

  • Markdown first. The output is a heading-and-list document a model can read directly. Measured on a ten-result search, the Markdown form is 32 % fewer tokens than the equivalent JSON (see the measurements).

Related MCP server: webfetch

Quick start

This is the fastest path that works today. It needs no npm publication, because it builds the package from source and links the free-search-mcp-ts binary into your PATH:

git clone https://github.com/lhswgzy/free-search-mcp-ts
cd free-search-mcp-ts
npm install && npm run build && npm link
free-search-mcp-ts install

That registers the server with the MCP clients installed on this machine. Then confirm the engines and network work from here, and try it without involving a client at all:

free-search-mcp-ts doctor
free-search-mcp-ts search "state of the art in retrieval augmented generation"
free-search-mcp-ts research "how does reciprocal rank fusion work" --depth 2

Restart your client afterwards. If you would rather see what the installer would do before it touches anything, free-search-mcp-ts install --dry-run prints the plan without writing.

Once the package is published to npm

npx -y free-search-mcp-ts is the intended path for anyone who does not want a local clone, but the package is not on npm yet. Once it is, every command above becomes:

npx -y free-search-mcp-ts install
npx -y free-search-mcp-ts doctor
npx -y free-search-mcp-ts search "state of the art in retrieval augmented generation"
npx -y free-search-mcp-ts install --dry-run

Tools exposed to the model

Tool

What it does

web_search

Searches several engines at once, fuses the results with RRF, de-duplicates and re-ranks. Use it when you do not yet know which page holds the answer.

research

The whole loop in one call: search, derive follow-up queries from the vocabulary the first results actually use, fetch the most relevant and diverse sources, then extract the passages that answer the question. Returns a citable Markdown brief.

fetch_url

Fetches one page or document and returns clean Markdown. Handles HTML, PDF, DOCX, XLSX, PPTX, EPUB, ODT, CSV, JSON and plain text. Long documents are truncated with a continuation offset instead of being refused.

fetch_urls

Fetches up to twelve URLs concurrently, reporting failures per URL.

parse_document

Parses a local file or a remote document: spreadsheets become Markdown tables, presentations become one section per slide, DOCX keeps headings, lists and tables.

search_index

Full-text search over everything fetched so far, using a local SQLite FTS5 index with CJK bigram support. Costs no network requests.

local_index

Reports on, prunes, clears or compacts that local index.

list_engines

Lists every engine with its tier, whether it needs a key, and its live health.

Every tool accepts format: "json" for callers that want to post-process the output instead of reading it.

Engines

Twenty-four engines, in tiers. The tier decides when an engine runs; you can always override with engines: ["bing", "wikipedia"] or ask for everything with engines: ["all"].

Primary — the documented keyless defaults

Engine

Notes

duckduckgo

HTML endpoint with an automatic fallback to the lite endpoint; region, freshness and safe-search parameters.

mojeek

An independent index with its own crawler — useful for escaping the Bing/Google duopoly.

googlenews

Google News RSS: articles rather than a general web index, with when: recency operators.

Fallback — used automatically when the primary tier returns too little

Engine

Notes

bing

Keyless HTML. Reachable on networks where DuckDuckGo and Google are not, which is what makes it the global fallback.

baidu, sogou, so360

Chinese-language engines, eligible for CJK queries or an explicit region: "cn". Their redirect wrappers are decoded locally.

Subject indexes — selected automatically from the query

wikipedia, hackernews, github, stackexchange, arxiv, openalex, crossref, npm, crates

A question about a Rust crate quietly gains crates.io; a question containing an error message gains Stack Overflow. The trigger list lives in src/engines/registry.ts and is plain readable regex.

Optional and keyed

Engine

Notes

startpage, brave

Keyless HTML front ends. Frequently behind a consent or anti-bot wall, which is detected and reported as a block rather than as "no results".

searxng

Point it at your own instance with SEARXNG_URL. Accepts a comma-separated list and fails over between them.

brave-api, serper, tavily, exa, google-cse

Optional API-key engines. When a key is present they join the first tier and get a higher RRF weight, because a documented JSON API beats a scraper.

BRAVE_API_KEY=... npx -y free-search-mcp-ts search "..."      # or SERPER_API_KEY, TAVILY_API_KEY, EXA_API_KEY
SEARXNG_URL=https://searx.example.org npx -y free-search-mcp-ts search "..."

How it works

Reciprocal Rank Fusion. Each engine contributes 1 / (k + rank) for every document it returns, with per-engine weights and a small bonus when several engines agree. RRF needs no calibration between providers, which matters because a Bing relevance score, a Mojeek BM25 score and a Google News publish order are not comparable numbers. See src/rrf.ts.

Tiered escalation. The primary tier runs first. Only if it returns fewer than a threshold does the fallback tier run, and only then do the subject indexes that match the query. A query the primary tier answers costs two or three HTTP requests; a query on a filtered network still succeeds.

A circuit breaker that survives restarts. A provider that is blocked (403/429/captcha) is benched for ten minutes after a single occurrence; a provider whose host cannot be connected to at all is benched for five. That state is written to the local SQLite index, so a restarted server — or your next command-line search — does not pay the timeout again.

Redirect wrappers decoded offline. Baidu, Sogou, 360, DuckDuckGo and Bing all hand back /link?url= or /ck/a?u= wrappers. Those are decoded locally, which fixes both citation quality and cross-engine de-duplication; only wrappers that cannot be decoded cost a redirect-following request.

A local full-text index. Every fetched page is stored in SQLite with an FTS5 index over its title and body. CJK text is expanded into overlapping bigrams at index time, because unicode61 would otherwise treat a whole Chinese sentence as a single token and make it unsearchable.

Markdown extraction. linkedom parses, Mozilla Readability isolates the article, and Turndown renders GitHub-flavoured Markdown — headings, lists, tables, fenced code with language tags and absolute links. When Readability declines a page (a changelog, a spec, a table-heavy docs site) a conservative fallback strips the chrome from the body instead of giving up.

Measured, not claimed

Claim

Measurement

Markdown costs fewer tokens than JSON

On a ten-result search: 3,752 chars / ~938 tokens as Markdown versus 5,550 chars / ~1,388 tokens as JSON — 32 % fewer tokens.

Fetching a page returns far less than the raw HTML

MDN's Fetch API page: 150 kB of HTML → 5.4 kB of Markdown (96 % smaller, ~38,300 → ~1,378 estimated tokens). Bing's and Baidu's result pages measure 97 % and 99 %.

The circuit breaker pays for itself

On a network where DuckDuckGo and Google News are unreachable: first search 11.9 s, subsequently 1.3 s, because the dead engines are benched in between.

Reproduce the first two with npx tsx scripts/measure-savings.mts; the third with two consecutive free-search-mcp-ts search calls.

Clients

free-search-mcp-ts install detects and patches the following. Only clients whose config file already exists are touched, every write is preceded by a .bak copy, and only the one key the server owns is modified.

Client

Config file

Claude Desktop

claude_desktop_config.json

Claude Code

~/.claude.json

Cursor

~/.cursor/mcp.json

Windsurf

~/.codeium/windsurf/mcp_config.json

VS Code (Copilot agent mode)

.vscode/mcp.json or the user profile

Cline

cline_mcp_settings.json

Roo Code

mcp_settings.json

Zed

settings.json (context_servers)

Codex CLI

~/.codex/config.toml

Gemini CLI

~/.gemini/settings.json

opencode

~/.config/opencode/opencode.json

LM Studio

~/.lmstudio/mcp.json

Continue

~/.continue/config.json

npx -y free-search-mcp-ts clients                          # every supported client id
npx -y free-search-mcp-ts install --client cursor --dry-run
npx -y free-search-mcp-ts install --client claude-desktop,codex
npx -y free-search-mcp-ts uninstall --client cursor

Ollama does not speak MCP. The tool-calling loop has to live in the client, so point an MCP-aware front end at this server instead — run it over HTTP and add it in Open WebUI, LibreChat or any other MCP-capable UI:

free-search-mcp-ts serve --transport http --port 8765
# then add http://127.0.0.1:8765/mcp as a streamable-HTTP MCP server

Command line

free-search-mcp-ts                     # serve MCP over stdio (what a client runs)
free-search-mcp-ts install             # register with the clients found here
free-search-mcp-ts uninstall           # remove the registration
free-search-mcp-ts doctor              # check the network, every engine and the local index
free-search-mcp-ts search  "<query>"   # multi-engine search
free-search-mcp-ts research "<query>"  # search + read + extract, as a citable brief
free-search-mcp-ts fetch   <url>       # one page or document as Markdown
free-search-mcp-ts parse   <file>      # a local document as Markdown
free-search-mcp-ts engines [--check]   # list engines, tiers and health
free-search-mcp-ts cache   <stats|search|clear|prune|vacuum|path>
free-search-mcp-ts config              # the effective configuration
free-search-mcp-ts tools [--json]      # the MCP tool manifest
free-search-mcp-ts clients             # supported client ids

Useful flags: --json, --max <n>, --engines a,b, --freshness day|week|month|year, --lang, --region, --depth 1-3, --sources <n>, --data-dir <path>, --verbose.

The subcommands call the same handlers the MCP server registers, so free-search-mcp-ts search "…" is a genuine end-to-end test of what a model would receive.

Configuration

Everything is optional. Copy .env.example to ~/.free-search-mcp/.env, or set the variables in your client's env block.

Variable

Default

Purpose

FREE_SEARCH_ENGINES

auto

Comma-separated engine ids, or auto for the tiered policy.

FREE_SEARCH_MAX_RESULTS

12

Default result count.

FREE_SEARCH_TIMEOUT

15000

Per-request timeout in milliseconds.

FREE_SEARCH_CACHE

1

Set to 0 to disable the local SQLite index.

FREE_SEARCH_CACHE_TTL_HOURS

24

How long a fetched page stays fresh.

FREE_SEARCH_PROXY

—

HTTP(S) proxy, e.g. http://127.0.0.1:7890. Standard HTTPS_PROXY is also honoured.

FREE_SEARCH_RESPECT_ROBOTS

1

Set to 0 to ignore robots.txt when fetching.

FREE_SEARCH_ALLOW_PRIVATE

0

Set to 1 to allow loopback/private addresses (off by default as an SSRF guard).

FREE_SEARCH_REGION, FREE_SEARCH_LANGUAGE

—

Region and language hints, e.g. us / en, cn / zh.

FREE_SEARCH_DATA_DIR

~/.free-search-mcp

Where the index and configuration live.

FREE_SEARCH_LOG_LEVEL

warn

silent, error, warn, info or debug.

A JSON config file at ~/.free-search-mcp/config.json accepts the same keys in camelCase.

Behind a proxy or on a filtered network

The server works out of the box on the open internet. If your network filters some providers, either point it at a proxy:

FREE_SEARCH_PROXY=http://127.0.0.1:7890 free-search-mcp-ts search "..."

or leave it alone and let the tiered fallback do its job — the fallback engines were chosen because they are reachable from networks that block DuckDuckGo and Google. free-search-mcp-ts doctor prints a per-engine sweep so you can see exactly which engines work from where you are.

Security and privacy

  • Nothing leaves your machine except the search queries and the pages you ask for. There is no telemetry, no analytics, and no server component.

  • fetch_url is model-driven, so it is guarded. Loopback, link-local and private-range addresses are refused by default; set FREE_SEARCH_ALLOW_PRIVATE=1 if you deliberately want to read a local service.

  • robots.txt is honoured for fetch_url by default, and a disallowed path is reported as such rather than silently skipped.

  • Local file access is limited to regular files under 100 MB, and only through the explicit parse_document tool.

  • Nothing is written to stdout except JSON-RPC. Diagnostics go to stderr, so a misbehaving log line can never corrupt the protocol stream.

Requirements

  • Node.js 20.19+ to run.

  • Node.js 22.5+ (24+ recommended) for the local SQLite index, which uses the built-in node:sqlite module. On older runtimes the cache degrades to an in-process LRU and everything else keeps working — doctor tells you which backend is active.

  • No native modules, no compiler, no post-install build step.

Verification status

Honesty about what has been tested matters more than a feature list.

  • Verified against live traffic from this machine: Bing, Baidu, Sogou, 360, Hacker News (Algolia), GitHub, Stack Exchange, npm, crates.io, OpenAlex, Crossref, and the full fetch → Markdown → index → re-read pipeline. doctor's engine sweep and the CLI subcommands were exercised end to end, and the MCP server completed a 17-check protocol handshake (initialize, tools/list, tools/call, ping) over stdio.

  • Verified against captured real responses: arXiv, and every engine listed above.

  • Verified against fixtures only, because this network cannot reach them: DuckDuckGo, Mojeek, Google News, Startpage, Brave, SearXNG, Wikipedia. Their parsers, parameter mapping and error paths are covered by tests built from the documented markup and API shapes, but they have not been exercised against a live response. If one of them misbehaves for you, that is the most likely place, and an issue with the raw response attached is the fastest fix.

  • Not verified at all: the five keyed engines' live responses (no keys available here). Their request construction and response mapping are verified offline against fixtures.

Development

git clone https://github.com/lhswgzy/free-search-mcp-ts
cd free-search-mcp-ts
npm install
npm run build          # tsc -> dist/
npm test               # vitest
npm run typecheck
node dist/cli.js doctor

npm run verify runs the whole acceptance sequence — typecheck, build, the test suite, both MCP protocol probes, the CLI surface and a packaging leak check — and prints one summary. npm run verify:online additionally performs a live search, fetch and engine sweep. The two protocol probes are standalone scripts, so you can also check a build directly:

node scripts/probe-stdio.mjs     # speaks MCP over stdio like a client does
node scripts/probe-http.mjs      # exercises the streamable-HTTP transport

Layout:

src/
  cli.ts            command line and the stdio/HTTP entry point
  server.ts         MCP server, tool registration, both transports
  search.ts         tiered orchestration, RRF, wrapper decoding
  research.ts       search -> fetch -> passage extraction
  rrf.ts            Reciprocal Rank Fusion and de-duplication
  cache.ts          SQLite FTS5 page index and circuit-breaker state
  http.ts           proxy, retries, size caps, the SSRF guard
  robots.ts         robots.txt parsing and caching
  config.ts         defaults <- config.json <- .env <- environment
  html/markdown.ts  HTML -> Markdown pipeline
  fetch/page.ts     the fetch-and-convert service
  fetch/documents.ts PDF, DOCX, XLSX, PPTX, EPUB, ODT, CSV, JSON
  engines/          one file per engine, plus kit.ts and registry.ts
  install/          client specs and the installer
  tools/            tool schemas, handlers and Markdown renderers
tests/              vitest suites and engine fixtures

Adding an engine is one file plus one line in src/engines/registry.ts. Every engine declares its own tier, RRF weight and, for subject indexes, the query patterns that make it eligible.

Limitations

  • research() quotes, it does not summarise. This server has no language model, so the brief is extractive by design: it selects and quotes the passages that match the query and labels every one with a citable source. The calling model writes the synthesis. Calling that a summary would be a lie.

  • HTML scraping is inherently fragile. Engines change their markup. The structural fallback extractor adapts without selectors, and a broken engine degrades to zero results and gets benched rather than breaking the search — but a provider can still change its markup faster than a release cycle.

  • The subject indexes are narrow by design. Wikipedia, GitHub, crates.io and arXiv index specific corpora, not the web.

  • No OCR. A scanned PDF is reported as having no extractable text rather than silently returning nothing.

Origin and authorship

Author: lhswg — GitHub lhswgzy — © 2025, MIT. Source of the idea: sweetcornna/free-search-mcp, a Python project (MIT, on PyPI as free-search-mcp).

This codebase was written from scratch. It is not a fork; it shares no commit history, no source file and no engine parser with that project, and nothing from it was read while this was written. The npm package is named free-search-mcp-ts rather than free-search-mcp deliberately, so that the original author keeps their own project name on their own registry. Bugs in the Python project should be reported there, not here. The full statement, including what is and is not claimed, is in AUTHORS.md.

License

MIT — see LICENSE. Copyright © 2025 lhswg.


If this saves you a search API bill, a star is appreciated. Issues with a raw engine response attached are the most useful kind of bug report.

Available Tools

8 tools
fetch_urlFetch a URL as MarkdownA
Read-only

Fetch one web page or document and return its main content as clean Markdown, with the navigation, adverts and cookie banners removed. Handles HTML (via a readability pass), PDF, DOCX, XLSX, PPTX, EPUB, ODT, CSV, JSON and plain text. Results are stored in a local SQLite index, so reading the same URL again is instant and the page becomes searchable offline with search_index. Long pages are truncated rather than refused: the response reports a next offset you can pass back to continue. robots.txt is honoured by default and private/loopback addresses are refused as a safety measure.

Inputs: url (required), format, offset, refresh, +4 more.

fetch_urlsFetch several URLsA
Read-only

Fetch up to 12 URLs concurrently and return each as Markdown under its own heading. Cheaper and faster than calling fetch_url repeatedly because the requests overlap. Failures are reported per URL instead of failing the whole call.

Inputs: urls (required), format, refresh, max_chars, +1 more.

list_enginesList search engines and their statusA
Read-only

Show every engine this server knows about: what it indexes, whether it needs an API key, which tier it belongs to, and its current health (a repeatedly blocked engine is benched for a while). Use it to pick explicit engines for web_search, or to diagnose an empty result list.

Inputs: format.

local_indexInspect or maintain the local indexA
DestructiveIdempotent

Report on, prune, clear or compact the local cache that stores fetched pages and recent engine responses. stats is safe and cheap; clear and prune delete data permanently.

Inputs: scope, action, format, max_pages, +1 more.

parse_documentParse a local file or document URLA
Read-only

Read a document from a local path or a URL and return its text as Markdown. Supports PDF, DOCX, XLSX, PPTX, EPUB, ODT, CSV, JSON, HTML and plain text. Spreadsheets become Markdown tables, presentations become one section per slide, DOCX keeps headings, lists and tables, and PDFs get their hyphenation and column artefacts repaired. Everything is parsed on this machine — no upload, no conversion service.

Inputs: url, path, sheet, format, +1 more.

researchResearch a question (search + read + organise)A
Read-only

One call that does the whole mechanical research loop: search several engines, derive extra queries from the vocabulary of the results, fetch the most relevant and diverse sources, then extract the passages from each that actually answer the question. Returns a Markdown brief with numbered, citable sources and verbatim quoted passages — it does NOT paraphrase, because this server has no language model; you write the synthesis from the evidence it gathers. Use this instead of chaining web_search and several fetch_url calls whenever you need more than a snippet. depth 1 = the query as written. depth 2 (default) = the query plus expansions derived from what the first page of results is actually about. depth 3 = broader expansion and more sources.

Inputs: query (required), depth, format, region, +9 more.

search_indexSearch the local page indexA
Read-only

Full-text search over every page this server has already fetched on this machine, using a local SQLite FTS5 index (with CJK bigram support). Use it to recall something you read earlier in the session, to avoid re-fetching, or to work offline. Returns cached page URLs, titles and highlighted snippets — and it costs no network requests at all.

Inputs: query (required), format, max_results.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.1/5.0

    Scored across 8 tools

    Disambiguation4/5

    Most tools have clearly distinct roles, and descriptions explicitly route between them (web_search vs research, fetch_url vs fetch_urls). The main overlap is fetch_url and parse_document, which both convert remote documents/HTML to Markdown from a URL, leaving some ambiguity about which to pick for a remote document.

    Naming Consistency4/5

    The dominant pattern is snake_case verb_noun (fetch_url, fetch_urls, parse_document, search_index, list_engines), which is predictable. Deviations are research (a bare verb/noun) and local_index (a noun phrase rather than an action), but these are minor.

    Tool Count5/5

    Eight tools is well-scoped for a search/fetch/research server, with each tool earning its place across search, fetch, parse, index recall and engine introspection. No redundancy or padding.

    Completeness5/5

    The surface covers the full lifecycle: multi-engine search, orchestrated deep research, single and batch fetching, document parsing, offline index recall, cache management and engine introspection. No obvious gaps or dead ends for the stated domain.

    Maintenance

    ActivitySlowing
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Local web search MCP server that fuses multiple search engines, fetches and compresses pages to reduce tokens and cost, with caching for repeated and paraphrased queries.
      66 PyPI
      61
      MIT
    • A
      license
      Not graded
      quality
      F
      maintenance
      MCP server enabling local-first web search, fetch, extract, and caching with citeable excerpts, no API key required. Supports research workflows for agents and apps.
      67 npm
      1
      MIT
    • A
      license
      A
      quality
      C
      maintenance
      Free web search, web-page reading as Markdown, and llms.txt documentation retrieval for MCP clients. Runs locally with Node.js over stdio using @alkasima/alka-search-mcp. Searches DuckDuckGo, Mojeek, and Brave; default providers require no API key. Website and provider availability may vary.
      3
      51 npm
      MIT