A
licenseNot graded
qualityB
maintenanceEnables AI agents to perform web searches and page fetching/crawling through self-hosted SearXNG and Crawl4AI, without depending on commercial search or scraping APIs.
MIT
private-search-4-agents
by blakemartz
fetch_url can retrieve public GitHub API URLs and convert the content to markdown for agent consumption.
Adds optional access to Kiwix offline corpora, enabling offline-serving and offline content capabilities when configured.
Uses a local Ollama LLM for cited answer synthesis, query expansion, and summarization when the LLM backend is configured.
Supports OpenAI-compatible LLM endpoints as an alternative backend for cited answer synthesis and summarization.
Exports OpenTelemetry traces and metrics for observability of the MCP server's search, fetch, and crawl operations.
fetch_url can retrieve Reddit threads as markdown.
Uses Redis/Valkey as a persistent shared cache for search results and fetched page content across clients.
Provides private web search through a self-hosted SearXNG instance, supporting multiple query phrasings, domain filters, time ranges, and configurable result content.
fetch_url can retrieve YouTube transcripts as markdown.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@private-search-4-agentssearch privately for self-hosted SearXNG setup guides and fetch the top result"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
private-search-4-agents
Private web search for AI agents, over MCP. Queries go to your own SearXNG; results are reranked by a local model; pages are fetched and extracted locally (fast paths, Crawl4AI, then an in-process raw fetch) into markdown, PDFs included. No third-party search or scraping API sees what your agents look up.
Forked from TadMSTR/searxng-mcp (MIT) at v3.30.1 and maintained separately: Firecrawl is gone, and the tool surface is tuned for agent consumption — few tools, small schemas, bounded and pageable output.
Quick Start
A running SearXNG instance is the only requirement. Everything else is optional and improves a specific dimension — see Prerequisites.
SearXNG only — nothing else deployed:
SEARXNG_URL=http://localhost:8081 \
CACHE_URL= RERANKER_URL= \
node build/src/index.jsWhat works in this configuration: search returns results, and fetch_url (and search with
content: "full") return page content as markdown via tier 3 (raw HTTP fetch + Readability,
wholly in-process), with PDFs extracted in-process too. What does not: semantic reranking
(results keep SearXNG's ordering, and highlights / fetch_url query passages fall back to
term overlap), query expansion and search answer: true (no LLM), caching (every call is live),
JS-heavy page rendering, and offline serving.
The startup capability line reports three states per capability, not two:
on (self-contained — nothing remote has to work for it to be true), unverified (configured,
but this process has never contacted the backend), and off (not configured). Nothing probes at
startup, so unverified is not a health failure — it's the honest absence of a health claim, not
an error. The command above prints exactly this:
capabilities on=tier3,pdf unverified=none off=tier2,cache,reranker,llm,kiwix,hister,solver,wayback,otel,natstier3 and pdf are the only things on, because they are the only capabilities that are true
without anything remote working — both run in-process. wayback is off until you set WAYBACK_ENABLED=true — it is a feature flag,
not a default.
The two overrides are what make it a clean minimal run rather than merely a working one.
CACHE_URL and RERANKER_URL both have non-empty defaults, so left alone the server spends the
whole run talking to services that were never deployed. Neither has a kill switch, so the lever
is an empty string.
The compose ladder
Four compose files in examples/, each a strict superset of the one above it. Start
at the rung that matches what you are willing to run; every rung's header comment states what
works, what does not, and which environment variable moves you up.
Rung | File | Adds | Containers | RAM | Unlocks |
1 | SearXNG + this server | 2 | ~600 MB | The least you can run | |
2 | cache, Crawl4AI | 4 | ~6 GB | JS rendering, real caching | |
3 | reranker | 5 | ~8 GB | Relevance reranking, | |
4 | Ollama, Kiwix, NATS, adblock | 9 | ~20 GB | Summarisation, offline corpora |
Rung 2 is the recommended starting point — the cache is the single largest latency win, and Crawl4AI gets you JavaScript rendering for one container.
cd examples
export SEARXNG_SECRET=$(openssl rand -hex 32)
docker compose -f compose.crawl4ai.yml up -dMoving up a rung is docker compose -f compose.<next>.yml up -d — all four share a compose
project name, so compose reconciles in place rather than starting a second copy of what you
already had.
As a container — this fork publishes no image; build the Dockerfile
(docker build -t private-search-4-agents .). The ghcr.io/tadmstr/* images referenced by
upstream docs and the compose examples are upstream's and carry the old tool surface — see
Deployment.
The reranker ships in this repo — cd docker/reranker && docker compose up, or pull ghcr.io/tadmstr/searxng-mcp-reranker. It is CPU-only, needs no API key, and has the model baked in so there is no cold start. See docker/reranker/README.md.
The adblock proxy is also published as an image, ghcr.io/tadmstr/searxng-mcp-adblock-proxy —
ad/tracker filtering for tier 3 (raw fetch). It is an open forward proxy with no authentication, so
loopback or a private network only — see docker/adblock-proxy/README.md
for placement. Its filter lists are fetched at runtime from ADBLOCK_FILTERS_URL (defaulted to
EasyList + EasyPrivacy) and reloaded on a schedule.
For a full local topology including Crawl4AI, Ollama, Kiwix, the adblock proxy, and NATS, see rung 4 of the ladder above — examples/compose.full.yml.
Related MCP server: Hermes SearXNG MCP Server
Why this server?
There are a number of SearXNG MCP servers. Most wrap the search endpoint and stop there. The differentiators here are in what happens after the search:
private-search-4-agents | Typical SearXNG MCP server | |
SearXNG search | yes | yes |
ML reranking of results | local cross-encoder, reorders by relevance — ships in | SearXNG's own ordering |
Full-page content retrieval | fetch cascade — Crawl4AI, in-process raw fetch + Readability → markdown (headings, code fences, links, tables), opt-in solver and Wayback | none, or a single raw fetch |
Long pages | pageable windows with an | truncated, silently |
PDF text | in-process extraction ( | none |
Per-domain routing | domain capability database learns which tier works per domain and skips the ones that do not | none |
Cited answers |
| none |
Site crawling |
| none |
Caching | persistent, shared across clients (Valkey/Redis) | in-process or none |
Challenge handling | detection-gated solver tier, plus a Wayback fallback | none |
Observability | OpenTelemetry traces and metrics, NATS events, structured logs | none |
Only SearXNG is required. Everything in the table above degrades gracefully: with nothing
else deployed, fetch_url still returns extracted content from the in-process tier 3, and the
startup capability line tells you exactly what's configured, what's configured but not yet
verified, and what's off — see Quick Start above.
Tools
Three tools, ~1.8k tokens of schema, plus MCP server instructions describing the flow.
Tool | What it does |
| SearXNG search with local reranking; up to 5 phrasings per call. |
| Read up to 5 public URLs as markdown — HTML, PDF, GitHub API, YouTube transcripts, Reddit threads. Returns a window with a footer saying how to continue ( |
| Crawl a site and return a URL/title/snippet manifest. Page content is cached, so follow-up |
clear_cache and domain_stats are operator tools, registered only with
SEARXNG_MCP_ADMIN_TOOLS=true. search_and_fetch and search_and_summarize were removed in
4.0.0 (use search with content: "full" / answer: true).
Full parameter reference: docs/tools.md.
Resources
Two read-only MCP Resources, additive to the tools above.
URI | What it returns |
| Effective configuration and capability state — which backing services are wired, the three-state capability line, behaviour switches and tunables, and which credentials are configured. |
| Aggregate view of the domain capability database, mirroring the aggregate mode of the admin |
config://searxng-mcp never includes a credential value. Secrets are reported as a
boolean — configured or not — and every URL is stripped of inline userinfo before it is
emitted, because Basic Auth in SEARXNG_URL and an inline password in CACHE_URL are both
supported. The payload is built from an allowlist rather than by dumping config and removing
known secrets, so a newly-added field cannot leak by default.
If you're behind an MCP proxy rather than talking to this server directly, verify it forwards
resources/*. Many proxies re-register upstream tools on their own server instead of
forwarding the wire protocol generically, and drop Resources silently as a result — the tools
still work, only the Resources go missing. See
Deployment for a worked example of checking
this.
Architecture
The fetch cascade, in full. Each stage is optional and skipped cleanly when unconfigured.
The llms.txt fast path (llms → llms_fetch below) matches against the host currently
allowlisted in domains.json — platform.claude.com — and extracts each page's section from
the document's url: front-matter (lowercase, YAML-style).
flowchart TD
entry["fetchPage(url)"]
cache{"Valkey cache hit?"}
cached["→ return cached { title, url, text }"]
github{"GitHub fast-path URL?\nraw · api · repo root · blob · tree · releases"}
gh_fetch["GitHub API / raw.githubusercontent.com / api.github.com\n→ return"]
llms{"llms.txt domain?"}
llms_fetch["Probe /llms-full.txt\nextract matching section\n→ return"]
kiwix{"Kiwix host?\nKIWIX_URL set"}
kiwix_fetch["Local Kiwix ZIM\nWikipedia · Stack Overflow · Arch Wiki\n→ cache + return"]
robots["robots.txt pre-check — tiers 2–3\ndisallowed → RobotsDisallowedError (cached 24h)"]
pdf{"PDF?\n.pdf path, or HEAD says application/pdf"}
pdf_fetch["Local PDF extraction — unpdf, in-process\npage-marked text · PDF_EXTRACT_ENABLED\n→ cache + return"]
tier_skip(["Per-domain tier skip\nsuccess rate <30% over ≥10 tries\nor tier_skip operator override"])
t2["Tier 2 — Crawl4AI\n$CRAWL4AI_URL · optional\ndirect — no adblock proxy"]
t3["Tier 3 — Raw HTTP + Readability\nfallback: raw HTML slice\nadblock proxy if $ADBLOCK_PROXY_URL"]
challenge(["Challenge detected\non this URL, this request?"])
solver["Solver — Byparr\n$SOLVER_URL · opt-in, SOLVER_ENABLED=true\nSSRF-guarded replay"]
t4["Tier 4 — Wayback Machine CDX API\narchived snapshot · WAYBACK_ENABLED=true"]
post["Post-extraction\nJSON-LD Article · title cascade\nog:title → twitter:title → title → h1 → URL"]
result["→ return { title, url, text }"]
entry --> cache
cache -->|hit| cached
cache -->|miss| github
github -->|yes| gh_fetch
github -->|no| llms
llms -->|yes| llms_fetch
llms -->|no| kiwix
kiwix -->|yes| kiwix_fetch
kiwix -->|no| robots
robots --> pdf
pdf -->|yes| pdf_fetch
pdf -->|no| tier_skip
pdf_fetch -->|"miss / error\n(not: no text layer)"| tier_skip
tier_skip --> t2
t2 -->|success| post
t2 -->|"empty / error"| t3
t3 -->|success| post
t3 -->|"empty / error"| challenge
challenge -->|"yes, SOLVER_ENABLED"| solver
challenge -->|no| t4
solver -->|success| post
solver -->|"miss / disabled"| t4
t4 -->|success| result
post --> result
style entry fill:#ffffff,stroke:#333333,color:#000000
style cache fill:#ffffff,stroke:#333333,color:#000000
style cached fill:#ffffff,stroke:#333333,color:#000000
style github fill:#dae8fc,stroke:#6c8ebf,color:#000000
style gh_fetch fill:#dae8fc,stroke:#6c8ebf,color:#000000
style llms fill:#dae8fc,stroke:#6c8ebf,color:#000000
style llms_fetch fill:#dae8fc,stroke:#6c8ebf,color:#000000
style kiwix fill:#fff9c4,stroke:#b8860b,color:#000000
style kiwix_fetch fill:#fff9c4,stroke:#b8860b,color:#000000
style robots fill:#ffffff,stroke:#333333,color:#000000
style pdf fill:#dae8fc,stroke:#6c8ebf,color:#000000
style pdf_fetch fill:#dae8fc,stroke:#6c8ebf,color:#000000
style tier_skip fill:#f5f5f5,stroke:#666666,color:#000000
style t2 fill:#d5e8d4,stroke:#5a8a4a,color:#000000
style t3 fill:#d5e8d4,stroke:#5a8a4a,color:#000000
style challenge fill:#f5f5f5,stroke:#666666,color:#000000
style solver fill:#d5e8d4,stroke:#5a8a4a,color:#000000
style t4 fill:#f8cecc,stroke:#a03030,color:#000000
style post fill:#e1d5e7,stroke:#7a5a8a,color:#000000
style result fill:#ffffff,stroke:#333333,color:#000000Read the rest — tier semantics, the domain capability database, adblocking, every fast path,
resilience and observability — in docs/architecture.md.
Documentation
Every environment variable, and setup for each optional backing service. | |
Full per-tool parameter reference. | |
Install, transports, HTTP auth, and MCP client recipes. | |
Fetch cascade, tier semantics, domain capability database. | |
SSRF, redirects, transport exposure, bounded reads, credentials. |
Index: docs/index.md.
Contributing
See CONTRIBUTING.md for setup instructions, commit conventions, and the PR process.
Integration tests
A real-Valkey integration suite covering domain-DB concurrency is gated on VALKEY_TEST_URL and skipped entirely when it's unset, so a plain pnpm test still works with no Valkey present:
VALKEY_TEST_URL=redis://:<password>@<host>:<port>/<scratch-db> pnpm testUse a scratch database index — the suite writes and deletes domain:* keys and refuses to run against index 0 or 1 as a safety guard.
License
MIT
Available Tools
3 toolscrawl_siteCrawl siteA
Map a site (sitemap, else link-following if enabled) into a URL/title/snippet list; pages are cached for fetch_url. A start URL with a path stays under it; include_path '/' crawls the whole site.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Site root or docs URL | |
| max_pages | No | Page cap (default 20) | |
| exclude_path | No | Skip URLs under this path prefix | |
| include_path | No | Only URLs under this path prefix; replaces the default scope | |
| same_domain_only | No | Stay on the start domain (default true) |
TDQS
A4.2/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and does disclose the discovery strategy, the return shape (URL/title/snippet list), a side effect (caching pages for fetch_url), and scoping rules by path. It omits failure behavior, robots/politeness handling, and whether the crawl is bounded by anything other than max_pages.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences with no filler, front-loading the core behavior and return shape before the scoping rules. Every clause carries information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 5-parameter tool with no output schema, no annotations, and rich schema coverage, the description covers purpose, output contents, caching side effect, and scope defaults. It falls short on crawl limits, failure modes, and the ambiguous 'if enabled' condition.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline would be 3, but the description adds real semantics: a start URL with a path constrains the crawl to that subtree, and include_path '/' widens it to the whole site. That explains the interaction between url and include_path/default scope in a way the schema fields alone only hint at.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource: it maps a site into a URL/title/snippet list, and it names the resolution strategy (sitemap first, else link-following). It also distinguishes itself from the fetch_url sibling by stating that crawled pages are cached for later fetch_url calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The crawl-then-fetch_url workflow is implied by 'pages are cached for fetch_url', which is useful context, but there is no explicit statement of when to crawl versus when to use search or fetch_url directly, and no prerequisites or exclusions. 'else link-following if enabled' also raises a condition (a setting being enabled) that the agent cannot verify from the schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fetch_urlFetch URLA
Read public URLs as markdown (PDFs included). GitHub files/repos/releases come from the GitHub API; YouTube and Reddit have fast paths; other pages go Crawl4AI (headless browser) then raw HTTP. Returns a window of the page with a footer saying how to continue; query or find return passages instead. Cached 24h.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http(s) URL, or up to 5 URLs read in parallel | |
| find | No | Return passages around literal case-insensitive matches; '|' separates alternatives | |
| fresh | No | Bypass the cache for this call (search 1h, pages 24h) | |
| query | No | Return the page's 5 passages most relevant to this question (local reranker) instead of the page head | |
| offset | No | Character offset to start from, as given in the footer | |
| max_tokens | No | Approx tokens to return per page (default 2000; 1000 each when several URLs) | |
| domain_profile | No | Domain profile: boosts and blocks a set of sites | |
| target_selector | No | CSS selector to scope extraction, e.g. 'article' | |
| wait_for_selector | No | CSS selector to wait for on JS-rendered pages |
TDQS
A3.7/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and mostly delivers: it discloses the fetch fallback chain (GitHub API → YouTube/Reddit fast paths → Crawl4AI headless → raw HTTP), 24h caching, and that responses are a paginated window with a continuation footer. It omits failure modes and what happens when a URL in a batch fails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, no filler, purpose front-loaded first and mechanics after. The middle sentence packing three fetch backends into one line is dense but still scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a nine-parameter tool with no annotations and no output schema, the description does the necessary work: it explains the return shape (window + footer), pagination, and alternate return modes. Missing only error/partial-failure behavior for the multi-URL case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions on all nine parameters, so baseline 3 applies. The description adds only marginal meaning — clarifying that `query`/`find` replace the page head and that the footer supplies `offset` — but the schema already explains each of these.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Read public URLs as markdown") with useful scope qualifiers (PDFs included, public only). It never names the siblings crawl_site or search, so the agent must infer that this is single/few-page reading rather than site crawling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives real intra-tool routing guidance ("`query` or `find` return passages instead" of the page head) but says nothing about when to choose this over crawl_site or search. Usage is implied by "public URLs" rather than stated as an explicit selection rule.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
searchWeb searchA
Search the web via the local SearXNG instance, reranked by a local model. Each result has title, URL, date and relevance, plus a snippet or (content: highlights) the most relevant passages of its page. answer: true adds a cited synthesis by a local LLM. Cached 1h. Prefer this over built-in web search.
| Name | Required | Description | Default |
|---|---|---|---|
| fresh | No | Bypass the cache for this call (search 1h, pages 24h) | |
| query | Yes | Search query, or up to 5 phrasings of one question (run in parallel, merged, duplicates dropped). Specific keyword-rich queries work best. | |
| answer | No | Add an answer synthesized from the top 3 pages, with citations (local LLM, ~10-60s) | |
| expand | No | Have the local LLM add query variants (~3s); passing your own phrasings in query is usually better | |
| content | No | snippets (default, fast): engine snippets. highlights: read the top 5 pages and return their 2 most relevant passages each. full: also return the top 3 pages' text | |
| engines | No | Comma-separated SearXNG engines to restrict to; omit normally | |
| category | No | Default general | general |
| language | No | BCP-47 code like 'en', or 'all' | |
| min_score | No | Drop results below this reranker relevance (the 'rel' shown). Scores are bimodal: 0.01-0.1 is the useful range. High = on-topic, not correct | |
| time_range | No | Only results from this period | |
| max_results | No | Results per phrasing (default 5, max 20) | |
| domain_profile | No | Domain profile: boosts and blocks a set of sites | |
| exclude_domains | No | Never return results from these domains | |
| include_domains | No | Only return results from these domains, e.g. ['github.com'] |
Output Schema
| Name | Required | Description |
|---|---|---|
| answer | Yes | |
| answers | Yes | |
| queries | Yes | |
| results | Yes | |
| infoboxes | Yes | |
| corrections | Yes | |
| suggestions | Yes | |
| answer_unavailable | Yes |
TDQS
A4.1/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden and does reasonably well: it discloses local SearXNG sourcing, local-model reranking, a 1h cache, and approximate latency for the answer path. It omits failure modes, rate limits, and what happens when a query returns nothing, which keeps it short of a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action, then progressively adds result shape, the answer option, and caching. A bit dense with parenthetical parameter callbacks, but every sentence carries information an agent would otherwise have to infer.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter tool with an output schema and no annotations, the description covers purpose, result structure, cache behavior, and LLM latency well. The many tuning parameters (engines, category, domain_profile, min_score) are fully documented in the schema, so the remaining gap is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains every parameter in detail. The description mostly echoes schema content ('content: highlights', 'answer: true') rather than adding new meaning, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource plus the implementation path: 'Search the web via the local SearXNG instance, reranked by a local model.' The reranking and result-shape details make it unmistakably a discovery tool, distinguishable from fetch_url and crawl_site which operate on known URLs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Prefer this over built-in web search' gives explicit routing advice, and 'Cached 1h' plus the fresh parameter signals when to force a new call. However, it never states the inverse relationship with its actual siblings (fetch_url, crawl_site) — i.e. when to stop searching and fetch a page instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
3 tool updates
v4.0.0- First observed
crawl_site - First observed
fetch_url - First observed
search
TDQS
A4.1/5.0
Scored across 3 tools
Disambiguation5/5
Each tool has a clearly distinct role: `search` discovers URLs, `fetch_url` retrieves a specific page's content, and `crawl_site` maps a site's structure. There is no meaningful overlap in their primary purposes, and the descriptions reinforce the boundaries.
Naming Consistency4/5
`fetch_url` and `crawl_site` follow a consistent verb_noun snake_case pattern, while `search` is a bare verb. This is a minor deviation, but all names are readable and use the same casing style.
Tool Count5/5
Three tools is a well-scoped set for a web retrieval server: discovery, single-page fetching, and site mapping. Each tool earns its place without redundancy.
Completeness4/5
The surface covers the core web research lifecycle: search, fetch, and crawl. Minor gaps exist, such as no explicit multi-URL batch fetching or structured extraction, but agents can work around these with the available tools.
Maintenance
ActivityActive
ResponsivenessNo issues
Related MCP Connectors
LLM-ready web search + instant answers + URL-to-clean-text fetch for agents and RAG.
Web search, fetch, extract, and research for AI agents. Markdown output + AI-synthesized answers.
Web search and page reading for AI agents, including JSON APIs, RSS feeds and YouTube.
Agent-native search engine with live web research optimized for AI agents.
Related MCP Servers
- AlicenseAqualityCmaintenanceEnables privacy-preserving web search and content extraction through a self-hosted SearXNG instance, aggregating 70+ engines and converting pages to Markdown for LLM use.3MIT
- AlicenseAqualityBmaintenanceGives AI agents local web search and page reading with no API keys, layering a SearXNG-backed search cache with browser-based fallbacks and returning clean page text from HTML, PDF, JSON, and archived snapshots. Fetches run in Docker alongside the search engine with SSRF protection, egress filtering, and untrusted-content marking so agents can safely retrieve live web data.2MIT
- FlicenseNot gradedqualityAmaintenanceEnables AI agents to run parallel web searches through SearXNG and/or You.com and crawl individual pages or entire sites into clean, deduplicated Markdown or JSON, including authenticated and documentation-focused crawls.-