searchpin
Searchpin is a self-hosted, free MCP server for web search and page fetching, designed for AI agents and Chinese network environments.
Run multi-engine web searches across Baidu, Sogou, Bing CN, and Bing Intl, with embedding-based semantic re-ranking.
Retrieve ranked titles, URLs, and snippets; control result count (1–20), general vs news vertical, publication freshness (day/week/month/year), and include/exclude domains.
Fetch a URL and extract clean readable text, removing ads, navigation, and boilerplate.
Extract text from PDFs locally with page labels; no OCR, and explicit errors for unreadable/encrypted/image-only PDFs.
Use diagnostic fields for HTTP errors, off-topic/dictionary results, engine pacing, cooldowns, and retry timing.
Support iterative research workflows: search → read snippets → fetch best URLs → refine and cross-verify.
Searchpin
Self-hosted web search for AI agents — zero API keys, zero cost. In 2026, the center of gravity in AI development is shifting from "chatting" to "autonomous task execution" — locally deployed, long-running agents are becoming the norm. When an agent runs 24/7, every web search must not be interrupted by API quotas or billing. Searchpin was designed for this from day one: zero external dependencies, zero usage limits. Agents can search, fetch, and verify without restriction, and developers never worry about cost.
Why Searchpin
🇨🇳 Optimized for Chinese Network Environments
Defaults to Baidu, Sogou, Bing CN, and Bing Intl — four search engines queried in parallel. Works natively within China's network, no proxy or VPN needed. Most overseas alternatives rely on Google, DuckDuckGo, or Brave, which are largely inaccessible inside China.
One running server sends Baidu searches at least 20 seconds apart. A Baidu captcha pauses Baidu for 10 minutes. A Sogou challenge pauses Sogou for 60 seconds, then 120 seconds on consecutive challenges. Other engines remain eligible; a search skips a paused engine without waiting for it. In web_search, search_pages[].engines reports which engines ran or were skipped, and retry_after_seconds reports the local time remaining. The next eligible search tests recovery; the timer does not guarantee that the upstream engine has unblocked access.
🧠 Semantic Re-ranking — a Differentiator Few Offer
Results from all four engines are not simply concatenated. They are merged and re-ranked by an embedding model based on semantic similarity to the query. What your AI receives is a curated list of high-quality results, not a pile of noisy links. Among free MCP search servers, very few offer this capability.
💰 Completely Free, Zero Barrier
No account registration, no API key application, no usage limits. No dependency on any commercial API — no risk of sudden paywalls or quota restrictions. The entire pipeline runs on your own machine.
🔍 Built for Modern Websites
Built-in SSR content extraction can parse pages rendered by Next.js, Nuxt, and similar frameworks, and extract JSON-LD structured data and microdata. Plain HTML scraping gets nothing from these sites.
web_fetch also detects PDFs by content type or file signature and extracts page-labelled text locally. Unreadable, encrypted, or image-only PDFs return an explicit error instead of binary garbage; OCR is not included. Extraction is limited to 25 MiB, 200 pages, and 2 million text characters.
🛡️ Pollution Detection + Cross-Verification
Automatically detects and flags results unrelated to your query. Four independent search engines provide cross-verifiable results, enabling your LLM to corroborate information across sources for more credible answers.
⚡ Deliberate Engineering Trade-offs
Every design decision was made with real-world usage in mind:
Token-conscious — Search results return only titles, URLs, and snippets. Structured extraction data is compact and truncated. Your LLM decides which pages are worth fetching in full, without wasting context window.
Fast response — Four engines queried asynchronously in parallel. Total time depends on the slowest engine, not the sum of all four. A typical search completes in 1–2 seconds.
Memory-friendly — The embedding model (~118MB) is downloaded once through hf-mirror.com (HuggingFace mirror for China), then reused from local cache.
Related MCP server: GroundRoute
Quick Start
pip install searchpin && searchpin-setupOn first run, the embedding model (~118MB) is downloaded once via hf-mirror.com (HuggingFace mirror for China). That is the only one-time setup.
Configuration
Claude Desktop / Cursor / any MCP client
Add to your mcpServers config:
{
"mcpServers": {
"Searchpin": {
"command": "searchpin-server",
"args": []
}
}
}VS Code
Or manually, add to .vscode/mcp.json:
{
"servers": {
"Searchpin": {
"command": "searchpin-server",
"args": []
}
}
}Docker
docker run -i --rm ghcr.io/telly6/searchpin:latestPython API
from searchpin import SearchEngine
engine = SearchEngine()
results = engine.search("Python 3.13 新特性")
page = engine.fetch("https://docs.python.org/3/whatsnew/3.13.html")
engine.close()Available Tools
2 toolsweb_fetchA
Fetch a URL and return clean, extracted text content (boilerplate, ads, and nav removed). Use to read articles, documentation pages, or API responses in full. PDF text layers are extracted locally with page labels; unreadable PDFs return an explicit error. OCR is not included. For broad queries, use web_search first, then fetch specific URLs from its results. After fetching, look for technical terms, names, or references you can use as keywords for a better follow-up search.
⚠️ FAILURE IS NORMAL — DO NOT GIVE UP AFTER 1-2 FAILED FETCHES:
If a fetch returns empty body, 403, or obvious JS-template garbage, try the NEXT URL from the search results immediately. Each retry costs only ~0.5s.
Common failure causes: JS-rendered SPA shells (stock pages, forums), Chinese paywalls/captchas (wenku.baidu.com, zhihu.com), government site timeouts.
Rule of thumb: try at least 3 URLs from 3 different domains before concluding the content is unreachable. Skipping this wastes the search you already paid for.
When retrying, pick a URL from a DIFFERENT domain — same-domain failures often share the same root cause (e.g., all pages behind the same Cloudflare/WAF).
RETRY SILENTLY: When a fetch fails, retry with the next URL immediately as another tool call. Do NOT narrate each failure to the user ('this one failed, trying another…'). Only mention failures in your final reply if ALL attempts were exhausted.
Inputs: url (required).
web_searchA
Search the web via multiple engines (Baidu, Sogou, Bing CN, Bing Intl). Returns ranked titles, URLs, and snippets (no full page content — use web_fetch to get the full text of any result). Baidu searches are spaced by at least 20 seconds per server process. A Baidu captcha pauses only Baidu for 600 seconds; other engines can still return results. If search_pages reports off_topic_headword, the source returned dictionary entries about only part of the query. Refine the search using distinct terms, another language or an official domain. HTTP errors are reported in http_status. In search_pages, paced means the local interval skipped a source; cooldown means it is temporarily paused. Results re-ranked by embedding similarity — the top results are the most relevant.
Iterative search is normal: search → read → refine → search → read → synthesize. Search returns only titles + snippets. Read snippets to identify promising results, then call web_fetch to get full page content for the URLs that look most useful.
BEFORE FETCHING, READ THE SNIPPETS FIRST:
Search results come with title + URL + snippet only (no full content). Read snippets to decide which URLs are worth fetching.
Fetch only the 1-3 most promising URLs — do not blindly fetch everything. Each unnecessary web_fetch wastes 1-3s per call.
⛔ WHEN RESULTS LOOK WRONG, DO NOT GIVE UP — ITERATE IMMEDIATELY: If search results are all irrelevant (wrong topic, wrong domain, generic encyclopedia entries, piracy sites, brand pages, file format tools, or dictionary entries), DO NOT conclude the information is unavailable. Try a more specific query or a different source. One possible cause is Bing's tokenizer splitting your query words into fragments that collide with unrelated content. Replace fragile/generic terms with INSEPARABLE identifiers — proper nouns, compound terms, subdomain names, or acronyms that the tokenizer CANNOT split. Then immediately re-search. Do not narrate the failure — just try a different formulation. The PRINCIPLE (not the specific domain) matters:
A law query: 'GDPR enforcement tracker 2026' not 'GDPR fine'
A car query: '全固态电池量产线' not 'solid state battery'
A finance query: 'USDJPY exchange rate' not 'yen dollar'
A programming query: 'rustc 1.87 changelog' not 'Rust release notes'
Keep site:, filetype:, and quoted phrases separate from neighbouring words.
SEARCH STRATEGY (patterns drawn from real-world use):
DATE TERMS HIJACK QUERIES — Putting year numbers or month names ("2026年", "June") directly in query text often backfires: the engine weights them as the primary topic and you get calendar summaries or current-events roundups instead of your actual topic. PREFER the 'freshness' parameter to control recency — it filters by publication date without polluting the query text.
SHORT COMMON WORDS COLLIDE — In Chinese, single characters that are also standalone dictionary headwords get tokenized independently and matched to dictionary/encyclopedia entries, regardless of surrounding context. In English, short common words that also name major products/services (credit card brands, consumer goods) cause the same problem. RECOGNISE THE PATTERN: if any word in your query could appear as a dictionary headword or a product name on its own, it WILL collide. Embed such terms inside longer inseparable compounds.
LONG PROPER NOUNS ARE ANTI-NOISE ANCHORS — Full institutional names, drug trade names, legal case codes, product model numbers resist tokenizer fragmentation. They act as unbreakable signals that pull results toward the right topic. When a shorter query goes off-target, try a version built around a specific long-form name.
LANGUAGE-SWITCH ESCAPES TOKENIZER TRAPS — When a Chinese query keeps hitting noise despite retries, switch the query to English. English-language media covers many China-specific topics, and the English tokenizer does not fragment CJK characters into dictionary entries.
REPLACE THE ANCHOR — Adding another word to a noisy query rarely fixes it (the original offending token still dominates the ranking). Instead, restructure the query around a DIFFERENT anchor term entirely — a proper noun, a model number, or a multi-word technical phrase that the tokenizer cannot split.
THE VOCABULARY BRIDGE — Chinese search engines match on LEXICOGRAPHIC (word-level), NOT semantic. The words you use MUST match how the target documents actually phrase things. This creates a critical gap:
User language (descriptive): 极端天气, 气候灾害, 高温热浪
Source language (operational): 暴雨橙警, 三级应急响应, 解除预警 These are two entirely different vocabulary systems and the engine CANNOT automatically map between them. When abstract category terms return only encyclopedia definitions and year-old retrospectives, switch to the OPERATIONAL terms that the institutions/sources themselves use — alert levels, government response codes, proper institutional names, numeric thresholds ("四十度"), or named entities ("台风蔷薇").
Abstract: '2026年极端天气气候灾害' → only dictionaries & retrospectives
Operational: '暴雨橙警 应急响应 中央气象台' → nmc.cn real-time alerts If a scoped search has no matches, the sampled results may not cover the document. Try a title, identifier or official document link when available.
⚡ NUMERIC DATA CROSS-VERIFICATION:
When a critical number appears in one result, confirm it against at least one other source before accepting it as fact. Different aggregators can lag; conflicting values demand a third source.
⛔ DO NOT STOP EARLY — PERSISTENCE IS REQUIRED:
When a search returns 10 irrelevant results, that does NOT mean the information doesn't exist. It means the current query formulation failed to reach it. Real-world testing shows: the same engine (baidu/bing_cn) that returned 100% calendar-dictionary noise for '2026年极端天气' returned nmc.cn real-time severe weather alerts (rerank 0.908) when reformulated as '暴雨洪涝 应急响应 中央气象台 预警'.
Top-10 failure is a QUERY PROBLEM, not a content-availability problem. The valid pages are likely at positions 11-30, pushed out of view by noise that matched your abstract terms. Do not conclude 'information unavailable' from a failed top 10.
EXHAUST AT LEAST 3-4 DISTINCT STRATEGIES before any 'not found' verdict:
Different vocabulary (descriptive → operational)
Different language (CN ↔ EN)
Different anchors (category term → proper name/number)
Site-restricted search (site:target-source.tld)
Never stop after 1-2 rounds just because the first attempts returned noise. The difference between 'completely unsolvable' and 'fully answered in 1 round' is often a single query reformulation — changing '极端天气' to '暴雨橙警'.
CONDUCT ALL ITERATIONS SILENTLY: When you need multiple rounds of searching and fetching to gather information, run all iterations as tool calls without narrating each step in your reply. Present only the final synthesized answer to the user. Do NOT say things like 'let me search again' or 'I tried searching for X but got Y, let me try Z instead' — just do it and deliver the result.
Inputs: query (required), topic, freshness, max_results, +2 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 2 tools
The two tools occupy clearly separate stages of the retrieval loop: web_search returns ranked titles/URLs/snippets, web_fetch returns full page text for a specific URL. Descriptions explicitly link them (search results tell you which URLs to fetch), so an agent cannot mistake one for the other.
Both names are lowercase snake_case with the same verb_noun shape (web_search, web_fetch) and share a 'web_' prefix, forming a tight, predictable pair. The naming directly mirrors the read/fetch distinction.
Only two tools, but they are exactly the minimal complete pair for a web research server — search without fetch or fetch without search would be useless. It is thin enough that it sits just above the borderline band, and nothing superfluous is included.
Search + fetch covers the full discover-then-read lifecycle, and the descriptions handle recency, multi-engine fallback, and snippet-first triage. Minor gaps remain: no parameter surface documented for site/domain restriction, news/date-scoped variants, or a batch-multi-URL fetch, so some workflows require extra round trips.
Maintenance
Related MCP Connectors
Web search for AI agents — one tool across 6 engines, routed to the cheapest + cached.
Free web search for AI agents. No API key required. Hosted MCP in active development.
Multi-engine search for AI agents. Trust scoring, local corpus, MCP-native. Self-hostable, BYOK.
The best web search for your AI Agent
Related MCP Servers
- AlicenseAqualityCmaintenanceAn advanced web browsing server enabling headless browser interactions via a secure API, providing features like navigation, content extraction, element interaction, and screenshot capture.627MIT
- AlicenseAqualityDmaintenanceWeb search for AI agents across 6 engines (Serper, Brave, Exa, Tavily, Firecrawl, Perplexity) through one search tool. Routes each query to the cheapest engine that clears a quality bar and caches repeats. Hosted, streamable-HTTP, BYOK supported.11MIT
- AlicenseAqualityFmaintenanceFree multi-source web search server for AI agents, with confidence scoring and token optimization.3Apache 2.0
- AlicenseAqualityCmaintenanceEnables web search via a real browser for any LLM, bypassing anti-bot measures without API keys.5MIT