The fetchaller server provides web scraping and search capabilities with automatic bot challenge bypass, offering specialized tools for Reddit, e-commerce, marketplaces, jobs, real estate, and more. Key tools:
fetch: Fetch any URL as clean markdown or raw HTML, supporting HTML, JSON, XML, CSV, and PDF; supports GET/POST, custom headers, token limits, and timeouts. Automatically solves bot challenges.
search: Combined Google and DuckDuckGo web search returning titles, URLs, and snippets with pagination.
Reddit: Browse subreddit listings (browse_reddit) and search posts globally or within a subreddit (search_reddit), with sorting and time filters.
AliExpress: Product details (get_aliexpress_product) and product search (search_aliexpress) with price filters, sorting, and pagination.
Alibaba: B2B product details including tiered pricing, MOQ, lead times, and supplier info (get_alibaba_product), and product search (search_alibaba) with price/sort filters.
search_marketplace: Search Kijiji, Craigslist, and Facebook Marketplace simultaneously using human-readable parameters (city, category, price range, condition); Kijiji auto-skipped for US locations.
LinkedIn Jobs: Public job search (search_linkedin_jobs) by keywords, location, experience, workplace type, salary, etc. (no account required); full job details by ID (get_linkedin_job).
realtor.ca: Search Canadian homes for sale or rent (search_realtor) with filters for price, beds, baths, property type, and more.
UniFi Manuals: Download Ubiquiti UniFi installation guides (get_unifi_manual) as readable PDF or PNG.
Additionally, it includes automatic challenge solving (Cloudflare, reCAPTCHA, etc.) and security features like SSRF protection and rate limiting.
Automatically bypasses Akamai bot challenges when fetching URLs.
Allows searching Alibaba.com B2B products and retrieving product details with tiered pricing, MOQ, and supplier info.
Allows searching AliExpress products with filters and retrieving product details including price, specs, ratings, and reviews.
Automatically bypasses Cloudflare bot challenges when fetching URLs.
Integrates DuckDuckGo web search as a supplement to Google search results.
Integrates Facebook Marketplace search for listings across categories and locations.
Integrates Google web search for querying and retrieving search results.
Allows browsing subreddit listings and searching Reddit posts with sort and time filters.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@fetchaller search Reddit for Claude Code tips"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Fetch websites in Claude Code without per-domain permission prompts. Includes web search, complete logged-out New Reddit support, challenge detection, and automatic solving for the supported challenge types below.
Claude Code's built-in WebFetch asks permission for every new domain and blocks Reddit entirely. fetchaller fixes both:
fetch: Read any URL — solves supported passive challenges and reports unresolved blocks as errors
search: Web search via Google + DuckDuckGo
browse_reddit: Browse subreddit listings (hot/new/top/rising)
search_reddit: Search Reddit posts globally or within a subreddit
search_marketplace: Search Kijiji, Craigslist, and Facebook Marketplace simultaneously with human-readable params (city name, category, price range)
search_realtor: Search Canadian homes on realtor.ca for sale or rent with full filters (location, price, beds, baths, property/building type, ownership)
search_linkedin_jobs / get_linkedin_job: Search LinkedIn's public logged-out job board (keywords, location, date posted, remote/hybrid/on-site, experience, job type, salary) and read full postings — no account needed
get_unifi_manual: Download a Ubiquiti UniFi installation guide as a PDF or per-page PNGs. Ubiquiti publishes no PDF and draws these guides as vector artwork, so they carry no readable text — this rebuilds the pages into a real document
get_aliexpress_product: AliExpress product details (price, specs, ratings, reviews)
search_aliexpress: Search AliExpress products with price filters and sorting
get_alibaba_product: Alibaba.com B2B product details (tiered pricing, MOQ, lead times, supplier info)
search_alibaba: Search Alibaba.com B2B products
Related MCP server: Spider MCP Server
# Clone and install
git clone https://github.com/Averyy/fetchaller-mcp.git
cd fetchaller-mcp
uv sync
# Point this at branded Google Chrome. The pinned Linux reference build in
# Dockerfile gives the closest browser/transport identity match.
export BROWSER_EXECUTABLE_PATH=/path/to/Chrome-for-Testing
# Add to Claude Code
claude mcp add fetchaller -- uv --directory "$(pwd)" run fetchaller-mcpAdd permissions to ~/.claude/settings.json:
{
"permissions": {
"allow": [
"mcp__fetchaller__fetch",
"mcp__fetchaller__search",
"mcp__fetchaller__browse_reddit",
"mcp__fetchaller__search_reddit",
"mcp__fetchaller__search_marketplace",
"mcp__fetchaller__search_realtor",
"mcp__fetchaller__search_linkedin_jobs",
"mcp__fetchaller__get_linkedin_job",
"mcp__fetchaller__get_unifi_manual",
"mcp__fetchaller__get_aliexpress_product",
"mcp__fetchaller__search_aliexpress",
"mcp__fetchaller__get_alibaba_product",
"mcp__fetchaller__search_alibaba"
]
}
}Restart Claude Code.
Add this to your project's CLAUDE.md (or global ~/.claude/CLAUDE.md) to instruct Claude to prefer fetchaller:
## Web Fetching & Search
**ALWAYS use fetchaller tools instead of WebFetch and WebSearch.** fetchaller has no domain restrictions and produces cleaner output.
- `mcp__fetchaller__fetch(url, maxTokens?, timeout?)` — Fetch any URL → clean markdown
- `mcp__fetchaller__search(query, page?)` — Web search (Google + DuckDuckGo)
- `mcp__fetchaller__browse_reddit(subreddit, sort?, time?, limit?)` — Browse subreddit listings
- `mcp__fetchaller__search_reddit(query, subreddit?, sort?, time?, limit?)` — Search Reddit posts
- `mcp__fetchaller__search_marketplace(query, location, platforms?, category?, sort?, condition?, min_price?, max_price?)` — Search Kijiji + Craigslist + Facebook Marketplace
- `mcp__fetchaller__search_realtor(location, transaction?, property_type?, building_type?, min_price?, max_price?, min_beds?, min_baths?, ownership?, sort?, page?)` — Search realtor.ca homes
- `mcp__fetchaller__search_linkedin_jobs(keywords, location?, date_posted?, workplace?, experience?, job_type?, min_salary?, sort?, start?, limit?)` — Search LinkedIn public jobs
- `mcp__fetchaller__get_linkedin_job(job_id)` — Full public detail for one LinkedIn posting
- `mcp__fetchaller__get_unifi_manual(url, format?)` — Rebuild a UniFi installation guide as `pdf`/`png`/`svg`. Writes into the server's `DATA_DIR`, so on a containerised deployment the files land inside the container
- `mcp__fetchaller__get_aliexpress_product(product_id, timeout?)` — AliExpress product details
- `mcp__fetchaller__search_aliexpress(query, page?, sort?, min_price?, max_price?, timeout?)` — Search AliExpress
- `mcp__fetchaller__get_alibaba_product(product_id, timeout?)` — Alibaba.com product details
- `mcp__fetchaller__search_alibaba(query, page?, sort?, min_price?, max_price?, timeout?)` — Search Alibaba.comThe mcp__fetchaller__fetch tool is now available:
# Fetch a URL
fetch https://example.com
# Fetch with token limit
fetch https://example.com maxTokens=10000
# Fetch slow site with longer timeout
fetch https://slow-site.com maxTokens=25000 timeout=60# Search the web
search "python asyncio tutorial"
# Page 2 of results
search "python asyncio tutorial" page=2Use search to find URLs
Use fetch to read them
fetch(url, maxTokens?, timeout?, raw?, method?, headers?, body?)Parameter | Type | Default | Description |
url | string | required | URL to fetch (http/https) |
maxTokens | number | 25000 | Max tokens to return |
timeout | number | 10 | Request timeout in seconds |
raw | boolean | false | Return raw HTML instead of markdown |
method | string |
|
|
headers | object | — | Extra request headers (max 32) |
body | string | — | Request body, |
Many search and listing APIs answer only to POST — Getro (which powers a large share of VC portfolio job boards), Algolia, GraphQL gateways. Their pages look static and fetch fine, so a GET-only fetcher reports an empty board on a site that appears to work.
{
"url": "https://api.getro.com/api/v2/collections/246/search/jobs",
"method": "POST",
"headers": { "Accept": "application/json" },
"body": "{\"hitsPerPage\":100,\"page\":0}"
}Content-Type defaults to application/json when the body is valid JSON.
A POST skips site-specific handling and the response cache — every site
interceptor answers a URL pattern by issuing its own request, which would
discard the caller's body, and the cache is keyed by URL alone.
Only GET and POST are accepted. PUT/PATCH/DELETE have no retrieval
use; connection and body-framing headers (Host, Content-Length,
Transfer-Encoding, …) cannot be set. Credentials (Authorization, Cookie,
…) are dropped if a redirect crosses to another host, and a 301/302/303
downgrades a POST to a bodyless GET as browsers do.
Clean markdown with:
Page title as H1
Scripts, styles, nav, footer, iframes removed (careers/jobs links in the site chrome are preserved — they are almost never in the body copy)
HTML converted to markdown
Redirects noted
Content truncated at token limit
A possibly_js_rendered warning when the page's content is assembled by
JavaScript, so an un-hydrated shell is not mistaken for an empty site — with
whatever the shell still declares (description, canonical URL, company
profiles from JSON-LD) recovered alongside it, since on such a page that is
the entire available answer
Scenario | Behavior |
Invalid URL | Error message |
Non-200 response | Error + partial body |
JSON content | Returned as-is |
XML/RSS feeds | Returned as-is |
CSV files | Returned as-is |
Plain text | Returned as-is |
PDF files | Text extracted |
Timeout | Error after timeout (default 10s) |
Huge page | Truncated at maxTokens |
search(query, page?)Parameter | Type | Default | Description |
query | string | required | Search query |
page | number | 1 | Result page (1-indexed) |
Searches Google (primary) and DuckDuckGo (supplement) in parallel. Returns titles, URLs, and snippets. Page 2+ queries Google only.
Two tools for Reddit research:
browse_reddit - Browse Subreddit Listingsbrowse_reddit({
subreddit: "LocalLLaMA", // without r/ prefix
sort: "hot", // hot, new, top, rising
time: "day", // hour, day, week, month, year, all (for "top" only)
limit: 10 // 1-25
})Returns post titles, scores, comment counts, and URLs. Use fetch to read full posts.
search_reddit - Search Postssearch_reddit({
query: "best mass spectrometry software",
subreddit: "labrats", // optional - limit to subreddit
sort: "relevance", // relevance, hot, top, new, comments
time: "year", // hour, day, week, month, year, all
limit: 10 // 1-25
})Returns matching posts with metadata. Use fetch to read full discussions.
Normal Reddit URLs and emitted links are canonicalized to www.reddit.com.
Structured JSON reads use the logged-out api.reddit.com origin and are
rendered as compact Markdown; HTML-only legs remain on the fixed New Reddit
www origin.
/r/{subreddit}/wiki/pages/ first reads the canonical New Reddit SSR page tree,
then falls back to New Reddit's own logged-out WikiPageRevisionsV2 page tree on
the fixed /svc/shreddit/graphql route when a community does not server-render
that tree. Both paths are anonymous: the page index needs no OAuth scope and no
browser. For a genuine historical collection URL, fetchaller
validates exact collection metadata from its archived New Reddit Redux snapshot
and hydrates every archived post ID from Reddit's current /api/info endpoint.
Archive shells, mismatched identities, empty collections, and missing current
posts are errors rather than successful empty or fabricated output.
Thread output includes the public vote score and upvote ratio returned by
Reddit, comment count and nested comments, outbound/discussion links, and
structured gallery, video, crosspost, poll, NSFW/spoiler/locked/archived, and
deleted-content state. Comment permalink URLs preserve the selected comment and
its parent context.
old.reddit.com input URLs are canonicalized to New Reddit; no normal request
depends on Old Reddit. Explicit .json URLs still return raw JSON. raw=true
on a normal Reddit URL returns the canonical New Reddit HTML representation.
Embedded author-written Old Reddit links are rewritten to their equivalent
www.reddit.com target.
Explicit JSON always remains parseable. When it exceeds maxTokens, fetchaller
retains only whole source scalars in a structural prefix and adds the top-level
JSON marker "_fetchaller_truncated": true (or a marker object at the end of a
top-level array). It never slices a string, substitutes a value, or reports
invalid JSON as success; a budget too small for both useful content and the
marker returns an explicit error.
fetchaller reads Reddit anonymously only and has no credential path: there is no OAuth flow, no client ID/secret, and no refresh or access token. Every Reddit read uses New Reddit's logged-out path. The wiki page index is served by the SSR tree or New Reddit's logged-out page-tree route. Routes Reddit serves only to a logged-in account — exact moderator rosters, and account-private upvoted/downvoted activity — return an explicit error saying so; no names or counts are ever guessed or reconstructed.
Vote data is not reconstructed: score is Reddit's public, fuzzed vote score,
and upvote_ratio is displayed only when Reddit returns it. Separate true
upvote/downvote counts are not available anonymously and are never invented.
The compact renderer requests limits and nesting depth from the caller's token budget, emits complete sections, and names every omitted section. Compactness is never allowed to remove public content silently. Any future size benchmark must ship its corpus, raw outputs, command, and timestamp with the result.
baselines/reddit-legacy-contract-v1.json is an independent, versioned
inventory derived from the last generic Old Reddit implementation. The
offline gate compares that contract with both the production router and
baselines/reddit-parity-corpus.json, then exercises route/schema/renderer/MCP
fixtures:
uv run pytest -q \
tests/test_reddit_legacy_contract.py \
tests/test_reddit_parity_corpus.py \
tests/test_reddit.pyThe separate live gate runs the corpus through a real stdio MCP server with its six-second default pacing (five-second minimum):
SMOKE_STDIO_COMMAND='["docker","run","--rm","-i","fetchaller-mcp:candidate","python","-m","fetchaller.main"]' \
uv run python scripts/reddit_parity.py \
--strict \
--include-unstable \
--output /tmp/fetchaller-reddit-parityThe runner starts with an empty dedicated wafer-cache directory, keeps it for
the warm and recreated-process stages, and records every body (including a
failed/challenge body), hash, command, timestamp, and outcome. For an exact
docker run command it mounts beneath entrypoint-owned /app/data, aligns the
runtime UID/GID with the host, validates a real owner-only Reddit cookie file
after warm and recreated stages, records non-secret Reddit cache hydration and
pure-HTTP verification counts, and rejects either recreated verification or
guarded-browser egress. A conflicting cache/ownership environment or --env-file
fails closed. When real OAuth credentials make the two OAuth corpus entries eligible,
Docker receives only --env NAME forwarding (never credential values in the
command or report); anonymous SSR output never counts as OAuth evidence.
Default runs require stable anonymous targets in --strict mode. The complete
release gate adds --include-unstable: the runner discovers current real
post/comment, wiki-revision, public-multireddit, live-update, and official
archived-collection IDs through separately recorded MCP calls, then requires
every public route
cold, warm, and recreated. It never substitutes a fake opaque ID.
Fixture-only targets always remain offline and can never become live evidence;
they are limited to inherently non-public access states such as private,
quarantined, banned, gated, forbidden, and not-found. A removed feature is not
a fixture-only waiver: its real public-read capability must still pass live.
Every route in the corpus is an anonymous public read and needs no
configuration. The live Reddit gates are currently waived on GitHub's hosted
runners only: Reddit answers the anonymous solve origin with 403 there, to a
real headful Chrome as well as to the HTTP client, which is consistent with
datacenter-IP blocking rather than anything request-side. Those runs carry an
explicit warning annotation so a green build is never mistaken for validated
Reddit, and the offline Reddit contract and fixture gate still runs on every
build. Every offline entry must carry a mandatory reason and complete fixture
evidence.
browse_reddit and search_reddit each use one JSON call. A normal fetch
uses one bounded JSON call for most URLs; a profile root uses five independent
public sources for metadata, activity, trophies, multireddits, and moderated
communities. Mapped fetch calls and both dedicated tools share the server's
request queue. Anonymous reads share a durable wafer session; configured API
reads share a separate cookie-isolated application session. Caller-selected
raw representations use the generic fetch path and its Reddit domain limiter.
Token refreshes and every API request use the same bounded request budget.
get_aliexpress_product(product_id, timeout?) - Product DetailsAccepts a numeric product ID (e.g., 1005006027485365) or full URL. Returns
price, specifications, ratings, and recent reviews via AliExpress's MTop API.
If MTop is unavailable after a live search, an exact matching validated search
listing can supply a clearly labeled, narrower title/price/rating snapshot.
The optional timeout is an end-to-end budget from 1 to 180 seconds (default 180).
search_aliexpress(query, page?, sort?, min_price?, max_price?, timeout?) - Search ProductsParameter | Type | Default | Description |
query | string | required | Search query |
page | number | 1 | Page number (1-indexed) |
sort | string | "default" | default, orders, price_asc, price_desc |
min_price | number | — | Minimum price filter |
max_price | number | — | Maximum price filter |
timeout | integer | 180 | End-to-end budget in seconds (1–300) |
get_alibaba_product(product_id, timeout?) - B2B Product DetailsAccepts a numeric product ID or full URL. Returns tiered pricing, MOQ, lead times, supplier info, and specifications. The optional timeout is an end-to-end budget from 1 to 300 seconds (default 180).
search_alibaba(query, page?, sort?, min_price?, max_price?, timeout?) - Search B2B ProductsParameter | Type | Default | Description |
query | string | required | Search query |
page | number | 1 | Page number (1-indexed) |
sort | string | "default" | default, price_asc, price_desc |
min_price | number | — | Minimum price filter (USD) |
max_price | number | — | Maximum price filter (USD) |
timeout | integer | 180 | End-to-end budget in seconds (1–300) |
search_marketplace — Search Kijiji, Craigslist, and Facebook MarketplaceSearches all three platforms concurrently with human-readable parameters and returns grouped results.
Parameter | Type | Default | Description |
query | string | required | Search keywords |
location | string | required | City name (e.g. "toronto", "st catharines, ON", "seattle") |
platforms | string[] | all | Platforms to search: kijiji, craigslist, facebook |
category | string | "all" | all, cars, electronics, furniture, clothing, tools, free, bikes, phones, motorcycles, boats, rvs, auto_parts, sporting, toys, baby |
sort | string | "date" | date, price_asc, price_desc, relevance |
condition | string | — | new, like_new, good, fair |
min_price | number | — | Minimum price in dollars |
max_price | number | — | Maximum price in dollars |
Kijiji is Canada-only and automatically skipped for US locations. Location matching supports exact names, common aliases (e.g. "niagara" → Hamilton CL area), and fuzzy matching for typos.
search_realtor — Search realtor.ca homesSearches Canadian homes for sale or rent via realtor.ca's api2, with the full filter set so an assistant can narrow a home search. Returns listings with price, address, beds/baths, size, agent, and a realtor.ca URL.
Parameter | Type | Default | Description |
location | string | required | City, neighbourhood, or postal code (e.g. "Ottawa", "Orleans, Ottawa", "M5V") |
transaction | string | "sale" | sale, rent |
property_type | string | "any" | any, residential, condo, recreational, vacant-land, multi-family, agriculture, parking |
building_type | string | — | house, duplex, triplex, townhouse, apartment, other |
min_price | integer | — | Minimum price (sale) or monthly rent |
max_price | integer | — | Maximum price (sale) or monthly rent |
min_beds | integer | — | Minimum bedrooms |
min_baths | integer | — | Minimum bathrooms |
ownership | string | — | freehold, condo |
sort | string | "newest" | newest, oldest, price-asc, price-desc |
page | integer | 1 | Result page (~20 per page, up to 600 returnable) |
Call fetch(url) on any listing URL for the full description, every property detail, and similar nearby homes. fetch also handles realtor.ca search/SEO/map pages (/{prov}/{city}/real-estate, /map) and all wellfound.com pages (startup job search, job detail, company profiles).
Validates URL (http/https only)
Blocks private/internal IPs (SSRF protection with DNS rebinding prevention)
Fetches with browser-like TLS fingerprints via wafer (Rust/BoringSSL) — rotates Chrome versions automatically
If bot challenge detected: solves automatically (see Bot Challenge Bypass below)
Detects content type
For HTML: removes junk elements (nav, footer, ads, cookie banners), applies site-specific cleanup (25+ sites including GitHub, Reddit, HN, Wikipedia, Medium, Stack Overflow, Amazon, eBay, AliExpress, Alibaba, DigiKey, Mouser, realtor.ca, wellfound.com, plus Ashby/Greenhouse/Lever/Gem/Dayforce/Cornerstone/Workday/BambooHR/JazzHR/Work-at-a-Startup job boards with embed + white-label detection, and more), converts to markdown
For JSON/XML/CSV/text: returns raw
For PDF: extracts text
Truncates to token limit
fetchaller detects bot challenges and attempts the supported solver. A request
is successful only after the resulting content is no longer a challenge or
block page; a Playwright navigation that remains blocked is an error, never a
successful fetch. First requests can be slower, while later requests can reuse
durably cached cookies. The timeout parameter is one total wall-clock budget
covering transport retries, queueing, and browser solving.
Challenge | Method | Speed |
Alibaba Cloud WAF (ACW) | Inline Python solver | ~1ms |
Alibaba Cloud WAF (TMD) | wafer browser solver | ~5-60s |
Cloudflare Managed Challenge | Patchright browser solver | ~3-30s |
Akamai Bot Manager | Patchright browser solver | ~3-15s |
Amazon rate-limit/CAPTCHA | Inline form parse + follow (no browser) | ~100ms |
DataDome, PerimeterX, Imperva | Patchright browser solver | ~3-10s |
Kasada | Browser CT token + Python SHA-256 PoW | ~3-10s |
GeeTest v4 slide CAPTCHA | CV notch detection + drag replay | ~5-15s |
hCaptcha | Checkbox auto-pass; image escalation is detect-only | ~5-30s |
reCAPTCHA v2 / Enterprise | Checkbox → ONNX image grid | ~5-60s |
F5 Shape, AWS WAF | Patchright browser solver | ~3-15s |
All challenge solving is handled by wafer's BrowserSolver (Patchright-based). Cookies are cached per-domain so subsequent requests skip the challenge.
Arkose/FunCaptcha and hCaptcha image escalation are detect-only and return an explicit challenge error. Vercel and generic-JS handling supports passive JavaScript checks only. An interactive CAPTCHA or any page that still contains the challenge is reported as unresolved.
Docker: The browser-complete image is linux/amd64 because Google does not
publish Chrome for Testing for Linux arm64. The build downloads one immutable
Chrome-for-Testing archive, verifies its SHA-256 digest and exact version, and
sets BROWSER_EXECUTABLE_PATH to that binary. BrowserSolver validates that the
configured executable is branded Chrome and warns if its version differs from
wafer/wreq's default emulation; browser-bound HTTP identity is aligned to the
installed browser, while the pinned image also keeps the TLS shape matched. The
local Compose file requests amd64 automatically on Apple Silicon, and the
cookie-data volume persists solved cookies across restarts.
Local (stdio): Browser support is included by default, but protected-site
solves require installed branded Google Chrome. Set BROWSER_EXECUTABLE_PATH
to that executable. Matching wafer's current emulation is recommended so the
browser and TLS identities stay aligned; the current Linux reference version
and archive checksum are the CHROME_VERSION and CHROME_SHA256 arguments in
Dockerfile.
src/fetchaller/content/ handles HTML→markdown conversion. Each site module exports is_<site>(url), SELECTORS_LIST, and optionally strip_<site>_junk(soup) / postprocess_<site>(markdown):
html.py — Generic pipeline + dispatch. Universal junk selectors, markdownify, whitespace cleanup. Generic JSON-LD Product fallback.
amazon.py — All TLDs (.com, .ca, .co.uk, .de, etc.). CSS selectors, soup cleanup, regex post-processors.
github.py — CSS selectors, URL transforms, file tree extraction, issue/PR/discussion extraction from embedded JSON.
reddit.py — Strict Reddit host recognition, New Reddit canonicalization,
public URL→structured routing (JSON plus the canonical SSR wiki page tree),
bounded compact renderers for threads/listings/profiles/rules/wiki, rich
media, and structured access states.
hackernews.py — CSS selectors, table unwrapping, story block reformatter.
medium.py — CSS selectors (data-testid), HTML-based detection for unknown custom domains.
huggingface.py — data-target attribute selectors, filter tag/button cleanup.
stackoverflow.py — All Stack Exchange sites. CSS selectors, soup cleanup, regex post-processors.
redflagdeals.py — RFD-specific CSS selectors, soup cleanup, regex post-processors.
forums.py — Generic forum support (XenForo, vBulletin, phpBB, Discourse). RSS/Atom feed autodiscovery.
wikipedia.py — CSS selectors for edit buttons, navboxes, TOC, reference lists.
alibaba.py — Embedded JSON extraction (window.detailData, window.__page__data_sse10), soup cleanup.
aliexpress.py — CSS selectors, soup cleanup, regex post-processors.
craigslist.py — All city subdomains. CSS selectors, regex post-processors. Search URL detection for SAPI intercept.
facebook_marketplace.py — URL detection only. GraphQL client in facebook_marketplace/ package.
digikey.py — All TLDs. CSS selectors, soup cleanup. Behind Akamai (wafer handles). HTML fallback without API key.
ebay.py — All TLDs. JSON-LD product extraction, search result DOM extraction (.s-item), regex post-processors.
molex.py — JSON-LD Product extraction (additionalProperty specs). CSR site — specs only in structured data.
mouser.py — All TLDs. CSS selectors, soup cleanup. Behind Akamai. HTML fallback without API key.
soylent.py — Shopify store cleanup, inventory extraction from gsf_conversion_data.
ti.py — Document viewer support for lazy-loaded datasheets.
ashby.py / greenhouse.py / lever.py / gem.py / dayforce.py / cornerstone.py / workday.py / bamboohr.py / jazzhr.py / workatastartup.py — Job-board platforms. All preserve the source's own field names, enum values, and section titles. Each posting and (where supported) board listing is dispatched to the platform's API/JSON shell before the generic HTML pipeline. Five embed/white-label detectors run during the HTML phase so company career pages like synaptivemedical.com/job-openings (white-label Dayforce), skywatch.com/careers/ (Ashby <script src="…/embed">), avidbots.com/company/careers/ (BambooHR <div id="BambooHR">), and earthdaily.com/job-openings (JazzHR multi-tenant) are upgraded to structured ATS output instead of returning empty SPA shells. See docs/site-apis.md for endpoints and detection details.
src/fetchaller/search/ — Google + DuckDuckGo combined. Result merging/dedup, 5-minute cache, CAPTCHA backoff. Uses wafer.AsyncSession(profile=Profile.OPERA_MINI).
CSR sites where HTML scraping produces garbage are intercepted in fetch_url() and routed to structured APIs:
Craigslist (src/fetchaller/craigslist/) — SAPI v8 client (sapi.craigslist.org). Up to 120 items/request with total count. Area IDs from page HTML, cached per hostname. Listing pages stay in HTML pipeline.
Kijiji (src/fetchaller/kijiji/) — Unauthenticated Apollo GraphQL (kijiji.ca/anvil/api). Search + listing detail. Prices in cents.
Facebook Marketplace (src/fetchaller/facebook_marketplace/) — GraphQL (facebook.com/api/graphql/). Geocoded search, listing detail with photos.
AliExpress (src/fetchaller/aliexpress/) — MTop API for products (token bootstrap + MD5 signing). SSR HTML for search. Reviews from feedback.aliexpress.com.
Alibaba.com (src/fetchaller/alibaba/) — SSR HTML with embedded JSON. No MTop API for international site.
eBay — SSR search results extracted from .s-item DOM elements, formatted as numbered list.
Mouser (src/fetchaller/mouser/) — Search API client. Requires MOUSER_API_KEY.
DigiKey (src/fetchaller/digikey/) — OAuth2 client_credentials API. Requires DIGIKEY_CLIENT_ID + DIGIKEY_CLIENT_SECRET.
Ubiquiti / UniFi (src/fetchaller/ubiquiti/) — *.store.ui.com, techspecs.ui.com, and ui.com/qig/<slug> installation guides. Store pages carry the price in their HTML but none of the technical specifications, which render client-side from __NEXT_DATA__; dispatch is by Next.js route, not URL shape. Guides are JS page assets of outlined vector art with no readable text, rebuilt into a PDF/PNG by get_unifi_manual.
Marketplace Search (src/fetchaller/marketplace/) — Unified orchestrator searching Kijiji, Craigslist, and Facebook Marketplace concurrently. Human-readable params mapped to platform-specific values. Auto-skips Kijiji for non-Canadian locations.
Dayforce HCM (src/fetchaller/content/dayforce.py) — Posting detail from SSR'd __NEXT_DATA__. Board listing via CSRF-protected POST to /api/geo/{namespace}/jobposting/search (NextAuth /api/auth/csrf round-trip required). White-label deployments on company domains are detected via __NEXT_DATA__.runtimeConfig.BASE_URL and rewritten to the canonical jobs.dayforcehcm.com board URL.
Cornerstone OnDemand (src/fetchaller/content/cornerstone.py) — SPA shell carries a JWT in csod.context. Posting from services/x/job-requisition/v2/requisitions/{reqid}/jobDetails; board listing POSTed to rec-job-search/external/jobs on the regional cloud host (us|eu|uk|au.api.csod.com).
Workday (src/fetchaller/content/workday.py) — {tenant}.wd{1-103}.myworkdayjobs.com boards and postings. Posting GET /wday/cxs/{tenant}/{site}/job{externalPath}; board POST /wday/cxs/{tenant}/{site}/jobs paginated in batches of 20 (capped at 200).
BambooHR (src/fetchaller/content/bamboohr.py) — {tenant}.bamboohr.com/careers. Board GET /careers/list; posting GET /careers/{id}/detail. Both return clean JSON unauthenticated. Widget embeds (<div id="BambooHR" data-domain="{tenant}.bamboohr.com">) on company sites are auto-detected and resolved to the tenant subdomain.
JazzHR (src/fetchaller/content/jazzhr.py) — {tenant}.applytojob.com/apply. Board SSR'd HTML (.list-group .list-group-item); posting reads schema.org JobPosting JSON-LD. Company sites that reference one or more JazzHR tenants via JS (e.g. earthdaily.com/job-openings) are auto-aggregated into a combined board.
All HTTP is handled by wafer (~/code/wafer). Fetchaller does NOT contain bot protection, challenge solving, or TLS fingerprinting code. If a site blocks requests, fix it in wafer.
Deploy fetchaller as a remote MCP server for Claude.ai, Claude Desktop, or any MCP client.
# Local loopback HTTP with authentication and restart-stable OAuth
export MCP_API_KEY='your-secret-key'
export JWT_SECRET="$(openssl rand -hex 32)"
export MCP_SERVER_URL='http://localhost:6000'
uv run python -m fetchaller.main --http
# Production Compose also requires a public HTTPS origin. It binds the app
# itself to 127.0.0.1:6000 for a host TLS reverse proxy.
export MCP_SERVER_URL='https://mcp.example.com'
docker compose up -dProduction traffic must terminate TLS before reaching the loopback-bound app.
Configure the host reverse proxy to forward to 127.0.0.1:6000. If the proxy
sets X-Forwarded-For, set TRUSTED_PROXY_IPS to only that proxy's exact
address or CIDR; untrusted forwarded headers are ignored.
# Build and test locally
docker compose -f docker-compose.local.yml up --build
# Test endpoints
curl http://localhost:6000/health
curl -X POST http://localhost:6000/mcp \
-H "Authorization: Bearer test-api-key-local" \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-d '{"jsonrpc":"2.0","method":"tools/list","id":1}'{
"mcpServers": {
"fetchaller": {
"type": "streamable-http",
"url": "https://mcp.fetchaller.com/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY"
}
}
}
}For Claude.ai web/mobile with cross-platform sync:
Go to Settings → Connectors → Add Custom Connector
Name: fetchaller
URL: https://mcp.fetchaller.com/mcp
Leave Client ID/Secret blank
Enter your API key when prompted
Variable | Default | Description |
| 6000 | Server port (1-65535) |
|
| Container supervisor deadline for HTTP readiness; expiry exits nonzero so Docker restarts the service |
|
| Grace period after supervisor |
| (required) | Bearer token for auth |
| loopback default outside production Compose | Public HTTPS origin for OAuth; exact loopback HTTP is allowed for local use |
| (required with API keys) | Stable signing secret for OAuth tokens — see below |
|
| Set to |
|
| Persistent OAuth client and refresh-token state |
|
| OAuth access-token lifetime in seconds (30 days) |
| 100 | Requests/minute per IP |
|
| Set to |
| — | Comma-separated addresses/CIDRs of reverse proxies whose rightmost |
| — | Mouser Search API key (free registration) |
| — | DigiKey API client ID (free registration) |
| — | DigiKey API client secret |
OAuth access tokens are stateless JWTs. When MCP_API_KEY is configured, HTTP startup fails
if JWT_SECRET is unset. For local HTTP development only, ALLOW_EPHEMERAL_JWT=1 opts into a
random per-process secret; every previously issued access token then becomes invalid when the
process restarts. Registered clients and hashed refresh tokens persist under DATA_DIR, while
short-lived authorization codes remain in memory.
Generate one value and keep it stable for the life of the deployment:
openssl rand -hex 32Pass it to the container the same way as MCP_API_KEY (compose environment:, Unraid
template variable, etc.). Verify it landed with printenv | grep JWT_SECRET inside the
container — an empty value counts as unset and prevents authenticated HTTP startup.
Two caveats:
Changing it invalidates every access token. Clients can normally recover with their persisted rotating refresh tokens, but rotating both the secret and OAuth state un-pairs every connector.
Do not derive it from MCP_API_KEY. Token payloads carry api_key_hash in cleartext,
so a derived secret is recoverable from any captured token, allowing forged tokens. Use an
independent random value.
Bearer auth with a raw MCP_API_KEY is unaffected by any of this — it is validated against
the environment variable directly and survives restarts.
SSRF Protection: Blocks localhost, private IPs, CGNAT/shared space, link-local addresses, and DNS rebinding services (nip.io, xip.io, etc.). Resolves hostnames, verifies every resolved address, and pins the connection to those addresses to close the DNS-rebinding window. Chromium challenge solving is covered too: every browser redirect and subresource is forced through a loopback-only SOCKS5 guard that re-applies the address policy and connects to an approved numeric IP. Public nonstandard ports remain supported; Chromium's own unsafe-port policy determines which ports a page may use. Transition addresses (NAT64, 6to4, Teredo, IPv4-mapped) are judged by the IPv4 they actually translate to, so 64:ff9b:: + a public host works on IPv6-only networks while 64:ff9b::7f00:1 (loopback) and 64:ff9b::a9fe:a9fe (cloud metadata) stay blocked. Unresolvable hosts fail closed but are reported as a DNS failure, not as a private-host block.
OAuth 2.1: PKCE required for authorization-code exchanges. Refresh tokens are hashed at rest and rotate on use.
Rate Limiting: Per-IP rate limiting with configurable limits.
fetchaller-mcp/
├── pyproject.toml # Python package config
├── src/fetchaller/ # Python source
│ ├── main.py # Entry point
│ ├── server.py # MCP server setup
│ ├── config.py # Configuration
│ ├── http/ # HTTP server (FastAPI)
│ ├── tools/ # MCP tools (fetch, search, reddit, aliexpress, alibaba, marketplace)
│ ├── content/ # Content processing (HTML→markdown, site-specific cleanup)
│ ├── search/ # Web search (Google + DuckDuckGo)
│ ├── aliexpress/ # AliExpress MTop API client, product, search, reviews
│ ├── alibaba/ # Alibaba.com product and search extraction
│ ├── mouser/ # Mouser Search API client
│ ├── craigslist/ # Craigslist SAPI client + location resolution
│ ├── kijiji/ # Kijiji GraphQL API client + location resolution
│ ├── facebook_marketplace/# Facebook Marketplace GraphQL client
│ ├── marketplace/ # Unified marketplace search orchestrator
│ ├── ubiquiti/ # UniFi store/techspecs specs + installation guides
│ ├── digikey/ # DigiKey API client (OAuth2 + product/search)
│ ├── cache/ # Response caching
│ ├── queue/ # Reddit rate limiting
│ └── security/ # SSRF, crypto, XSS
├── docker-compose.yml # Production deployment
├── docker-compose.local.yml # Local testing
├── Dockerfile # Container build
├── docs/ # Architecture & developer docs
├── CLAUDE.md # Instructions for Claude
├── README.md # This file
└── landing/ # Static site (fetchaller.com)
├── index.html # Landing page
└── llms.txt # LLM-readable project summary (llmstxt.org spec)wafer-py[browser] - HTTP transport with TLS fingerprinting, bot challenge bypass, and browser solver (Rust/BoringSSL + Patchright)
mcp - MCP protocol SDK
fastapi + uvicorn - HTTP server
beautifulsoup4 + markdownify - HTML to markdown
pymupdf4llm - PDF to markdown extraction
pyjwt - OAuth tokens
# Run tests
uv sync --extra dev
uv run ruff check src/ tests/
uv run pytest tests/ -x -q
# Test in Docker
docker compose -f docker-compose.local.yml up --build
curl http://localhost:6000/healthMIT
Browse a subreddit's posts. Returns metadata and URLs. Use mcp__fetchaller__fetch to read full post content.
| Name | Required | Description | Default |
|---|---|---|---|
| sort | No | Sort order | hot |
| time | No | Time filter (only applies to 'top' sort) | day |
| after | No | Pagination cursor from previous response | |
| limit | No | Number of posts (1-25) | |
| timeout | No | Request timeout in seconds (default: 10) | |
| subreddit | Yes | Subreddit name without r/ prefix |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must fully disclose behavioral traits. It mentions returns metadata and URLs, but does not state read-only nature, rate limits, authentication needs, error conditions, or side effects. This is a significant gap for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is extremely concise: two sentences, front-loaded with purpose, and immediately followed by actionable guidance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description mentions returns 'metadata and URLs', but not what metadata includes. No coverage of pagination, error cases, or tool-specific features. Adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (all 6 parameters described in schema), so baseline is 3. The description adds no additional meaning beyond schema; no explanation of parameter purpose or usage context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Browse a subreddit's posts' with specific verb and resource, and distinguishes from fetch by directing to use fetch for full content. However, it does not differentiate from search_reddit sibling, leaving ambiguity about when to use browse vs search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides partial guidance: it tells users to use fetch for reading full content, implying browse is not for that purpose. But it gives no explicit when-to-use vs siblings like search_reddit, and no prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch any URL and return the page content as clean markdown. Handles HTML, JSON, XML, CSV, and PDF files. Use this tool for reading/fetching web pages - it has no domain restrictions. For discovering URLs via search, use the search tool. For reading URL content, use this tool.
| Name | Required | Description | Default |
|---|---|---|---|
| raw | No | Return raw HTML instead of markdown (default: false) | |
| url | Yes | The URL to fetch | |
| body | No | Request body (POST only). Content-Type defaults to application/json when the body is valid JSON. | |
| method | No | HTTP method (default: GET). Use POST for search/listing APIs that only answer to POST (Getro, Algolia, GraphQL). POST requests skip site-specific handling and the response cache. | |
| headers | No | Extra request headers, e.g. {"Accept": "application/json"}. Connection and body-framing headers cannot be set. | |
| timeout | No | Request timeout in seconds (default: 10) | |
| maxTokens | No | Maximum tokens to return (default: 25000) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It mentions handling multiple file types and returning markdown, but omits details on caching, redirect handling, or rate limits. The schema adds some behavior (POST bypasses cache), but the description does not summarize this.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each adding critical information: core function, supported formats, and usage guidance against alternatives. No waste or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters and no output schema, the description covers core behavior, supported formats, and use cases. Missing details like error handling or timeouts, but schema covers parameter constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no parameter-specific meaning beyond what the schema already provides. The mention of 'handles HTML...' is about output, not parameters, and does not enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches any URL and returns markdown content. It differentiates from sibling tools by noting 'no domain restrictions' and explicitly contrasts with the search tool for URL discovery.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance to use this tool for reading web pages and directs users to the search tool for URL discovery. Does not contrast with other fetch-like siblings (e.g., get_alibaba_product), but the 'no domain restrictions' implies preference for generic use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get Alibaba.com B2B product details including tiered pricing, MOQ, lead times, supplier info, and specifications. Accepts a numeric product ID or full URL.
| Name | Required | Description | Default |
|---|---|---|---|
| timeout | No | End-to-end timeout in seconds (default: 180) | |
| product_id | Yes | Numeric product ID or full Alibaba.com URL |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It lists returned data types but does not disclose read-only nature, error handling, rate limits, or authentication needs. Adequate but with gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. Front-loads the main action and key details. Every piece of information is relevant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 2-parameter tool with no output schema and no annotations, the description is fairly complete. It explains what the tool does, what input to provide, and what kind of information to expect. Could mention that product_id can be just a number, but it's implied.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds value by clarifying that product_id accepts both numeric ID and full URL. For timeout, description adds nothing beyond schema defaults and bounds. Overall helpful beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Get Alibaba.com B2B product details' and lists specific data points (tiered pricing, MOQ, lead times, supplier info, specifications). Distinguishes from sibling tools like search_alibaba and get_aliexpress_product in scope and target platform.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly indicates when to use: when you have a numeric product ID or full URL. Does not provide when-not-to-use or name alternatives, but the context is clear for a retrieval vs. search tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get AliExpress product details including price, specifications, ratings, and recent reviews. Accepts a numeric product ID or full URL.
| Name | Required | Description | Default |
|---|---|---|---|
| timeout | No | End-to-end timeout in seconds (default: 180) | |
| product_id | Yes | Numeric product ID or full AliExpress URL |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions the tool retrieves details including price, specs, ratings, and recent reviews, which suggests a read-only behavior. However, it does not disclose potential failure modes, authentication needs, rate limits, or whether it fetches live data from the web, leaving some behavioral aspects unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long: the first states the purpose and output, the second states the input format. It is front-loaded with the core action and is free of unnecessary words or redundancies.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lists some returned fields (price, specifications, ratings, recent reviews) but does not provide a complete response structure. With no output schema, the agent must infer the full format. Additionally, it lacks details on constraints like pagination, language, or error handling, making it adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description repeats the input format ('numeric product ID or full URL') that is already in the schema description for product_id. It does not add significant additional meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get AliExpress product details including price, specifications, ratings, and recent reviews.' It uses a specific verb ('Get') and resource ('AliExpress product details'), and distinguishes itself from sibling tools like search_aliexpress by focusing on retrieving details for a known product rather than searching.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have a numeric product ID or full URL ('Accepts a numeric product ID or full URL'), but it does not explicitly state when to use this tool versus alternatives like search_aliexpress or get_alibaba_product. No when-not or alternative guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the full public detail for one LinkedIn job posting: description, seniority, employment type, job function, industries, and applicant count. Applying still requires a LinkedIn account.
| Name | Required | Description | Default |
|---|---|---|---|
| job_id | Yes | Numeric LinkedIn job ID (6-20 digits), the trailing number in a linkedin.com/jobs/view/<id> URL |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It notes that applying requires an account, but lacks information on rate limits, authentication needs, or any side effects. The tool is a read-only operation, but this is not explicitly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise with two sentences that front-load the core purpose and essential caveat. Every sentence is necessary and contributes to understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple input (one parameter) and no output schema, the description covers the key output fields and a practical limitation. However, it does not mention potential errors or response format, which could be useful for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% schema description coverage with a well-described job_id parameter. The tool description adds further context by explaining how to obtain the job ID from a LinkedIn URL, which enhances understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves full public detail for a single LinkedIn job posting, listing specific fields. It distinguishes itself from sibling search tools by being a detail fetcher, though not explicitly differentiating from get_alibaba_product or get_aliexpress_product which are similar in nature but for different platforms.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions that applying requires a LinkedIn account, which provides some context, but it does not guide on when to use this tool versus alternatives like search_linkedin_jobs. No explicit when-to-use or when-not-to-use guidelines are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search the web and return results with titles, URLs, and snippets. Use this to discover URLs, then use fetch to read full page content.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Result page (1-indexed, default: 1) | |
| query | Yes | Search query |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It mentions output format but lacks details on rate limits, pagination behavior beyond the schema, or error handling. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. Front-loaded with action and purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given schema coverage and no output schema, the description is fairly complete for a simple search tool. Could mention source engine or limitations, but adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers both parameters with descriptions (100% coverage). The description adds no additional meaning beyond schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it searches the web and returns specific result fields (titles, URLs, snippets), and distinguishes itself from specialized sibling tools by implying a general web search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides a clear usage pattern: use to discover URLs, then fetch for full content. However, it does not explicitly state when to avoid this tool in favor of specialized siblings, though the sibling list implies context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search Alibaba.com B2B products. Returns supplier listings with tiered pricing, MOQ, and supplier info.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-indexed, default: 1) | |
| sort | No | Sort order | default |
| query | Yes | Search query | |
| timeout | No | End-to-end timeout in seconds (default: 180) | |
| max_price | No | Maximum price filter (USD) | |
| min_price | No | Minimum price filter (USD) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It mentions the return type (supplier listings with pricing, MOQ, info) but does not disclose behavioral aspects such as rate limits, authentication needs, pagination behavior, or whether results are real-time. It is adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the primary action and key return info. There is no unnecessary content, and it is efficiently concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description covers the primary return elements (supplier listings, pricing, MOQ, supplier info). However, it omits details on pagination, result count limits, and how filters combine. The parameter schema is well-documented, but the description could be more complete for a tool with moderate complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no extra meaning beyond what the schema already provides (e.g., explanation of how min_price/max_price interact or sort options). It merely restates the purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Search', the resource 'Alibaba.com B2B products', and what is returned (supplier listings with tiered pricing, MOQ, and supplier info). It distinguishes from siblings like search_aliexpress (consumer) and search_marketplace (generic).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives, nor does it provide prerequisites or exclusions. The name and purpose imply B2B product search on Alibaba, but explicit guidance is absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search AliExpress products. Returns product listings with prices, ratings, and links. Best-effort — may fail if anti-bot protection triggers.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-indexed, default: 1) | |
| sort | No | Sort order | default |
| query | Yes | Search query | |
| timeout | No | End-to-end timeout in seconds (default: 180) | |
| max_price | No | Maximum price filter | |
| min_price | No | Minimum price filter |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It mentions best-effort execution and possible anti-bot failure, which is useful. However, it omits details on rate limits, data freshness, or whether results are real-time, which are important for a scraping tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: the first states purpose and return type, the second adds a behavioral caveat. No extraneous words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 6 parameters and no output schema, the description covers purpose and basic behavior but lacks details on pagination, sorting options in context, or filter usage. Schema descriptions for params help, but overall context is minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description does not need to add parameter details. The description adds no extra meaning beyond what the schema provides, maintaining the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Search' and resource 'AliExpress products', and specifies the return type (listings with prices, ratings, links). This distinguishes it from siblings like 'search_alibaba' (different platform) and 'get_aliexpress_product' (specific product).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for searching AliExpress and warns of potential anti-bot failure, but does not explicitly state when to use this tool over alternatives like 'get_aliexpress_product' or other search tools. No exclusions or context for when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search LinkedIn's public job board (no account needed). Filter by keywords, location, date posted, remote/hybrid/on-site, experience level, job type, and minimum salary. Returns title, company, location, posting date, and a linkedin.com/jobs/view URL. Call get_linkedin_job(job_id) for the full description.
| Name | Required | Description | Default |
|---|---|---|---|
| sort | No | LinkedIn's own sort is not honoured on this endpoint; "recent" sorts the fetched results by posting date here. | relevance |
| limit | No | Jobs to return (default: 10) | |
| start | No | Result offset for pagination (max 999) | |
| geo_id | No | LinkedIn geo ID, if you already have one. Takes precedence over location. | |
| job_type | No | Employment type | |
| keywords | Yes | Job title or skills, e.g. "software engineer" | |
| location | No | City/region, e.g. "Toronto, Ontario, Canada". Resolved to a LinkedIn geo ID automatically. | |
| workplace | No | Workplace type | |
| easy_apply | No | Only postings you can apply to inside LinkedIn (no external application form) | |
| experience | No | Experience level | |
| min_salary | No | Minimum annual salary filter | |
| date_posted | No | How recently the job was posted | any |
| under_10_applicants | No | LinkedIn's "Under 10 applicants" filter — strongly favours postings with few applicants so far |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description adds valuable context such as the sorting behavior ('LinkedIn's own sort is not honoured...') and the public nature. However, it does not disclose rate limits, read-only status, or potential changes to the endpoint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four concise sentences: stating purpose, listing filters, describing output, and referencing a sibling tool. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 13 parameters and full schema coverage, the description covers main functionality, return values, and sibling relation. It is mostly complete but could explicitly mention pagination behavior beyond what is in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description lists filter categories but does not add significant meaning beyond the schema descriptions; the 'no account needed' note is tool-level, not parameter-specific.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it searches LinkedIn's public job board, specifying 'no account needed'. It distinguishes itself from the sibling tool `get_linkedin_job` by noting that the latter provides full descriptions, creating a clear role separation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use this tool ('no account needed') and recommends a sibling tool for full descriptions, but does not explicitly exclude other sibling search tools or provide comprehensive when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search Kijiji, Craigslist, and Facebook Marketplace simultaneously. Takes human-readable parameters (city name, category, price range) and returns grouped results from all platforms. Use fetch(url) to get full listing details for any result URL.
| Name | Required | Description | Default |
|---|---|---|---|
| sort | No | Sort order | date |
| query | Yes | Search keywords (e.g. "golf r", "ikea couch") | |
| category | No | Category filter | all |
| location | Yes | City name, optionally with province/state (e.g. "toronto", "st catharines, ON", "seattle") | |
| condition | No | Item condition filter | |
| max_price | No | Maximum price in dollars | |
| min_price | No | Minimum price in dollars | |
| platforms | No | Platforms to search (default: all). Kijiji is Canada-only and auto-skipped for US locations. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It mentions 'grouped results' and simultaneous search, but omits details on rate limits, authentication, error handling, or result volume.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, immediately stating the main purpose and key features. No redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 8 parameters and no output schema, the description is very brief. It does not explain return format, pagination, error behavior, or how to interpret grouped results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds minimal value beyond what schema already provides. It restates 'human-readable parameters' but doesn't elaborate on format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Search' and the specific resources (Kijiji, Craigslist, Facebook Marketplace), distinguishing it from sibling tools like search_aliexpress or search_reddit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for multi-platform searches, but lacks explicit guidance on when not to use it or how it compares to alternatives like the 'fetch' tool for individual listings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search Canadian homes for sale or rent on realtor.ca with full filters (location, price, beds, baths, property/building type, ownership). Returns listings with price, address, beds/baths, size, agent and a realtor.ca URL. Call fetch(url) on a listing URL for the full description, every property detail, and similar nearby homes.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Result page (~20 per page, up to 600 total) | |
| sort | No | Sort order | newest |
| location | Yes | City, neighbourhood, or postal code (e.g. "Ottawa", "Orleans, Ottawa", "Toronto", "M5V") | |
| min_beds | No | Minimum bedrooms | |
| max_price | No | Maximum price (sale) or monthly rent | |
| min_baths | No | Minimum bathrooms | |
| min_price | No | Minimum price (sale) or monthly rent | |
| ownership | No | Ownership type | |
| transaction | No | Buy (sale) or rent | sale |
| building_type | No | Building type filter | |
| property_type | No | Property category | any |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool returns listings with price, address, beds/baths, size, agent, and a realtor.ca URL, and suggests using fetch for full descriptions. However, it does not mention any rate limits, authentication needs, or potential side effects, which would elevate it to a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loading the purpose and filter capabilities in the first sentence, and detailing return values and follow-up in the second. Every sentence earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of 11 parameters, no output schema, and no annotations, the description is fairly complete. It covers the tool's purpose, key filters, return fields, and a clear next step (fetch). It lacks only a mention of pagination beyond the page parameter description in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description summarizes key filters (location, price, beds, baths, property/building type, ownership) but adds little meaning beyond what the schema already provides. It does not elaborate on parameter usage nuances.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches Canadian homes for sale or rent on realtor.ca with full filters. It specifies the resource (realtor.ca) and the action (search), distinguishing it from sibling tools like search_aliexpress or search_alibaba.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives. It mentions a follow-up action (call fetch for details) but provides no guidance on selecting this tool over other search tools among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search Reddit posts. Returns metadata and URLs. Use mcp__fetchaller__fetch to read full post content.
| Name | Required | Description | Default |
|---|---|---|---|
| sort | No | Sort order | relevance |
| time | No | Time filter | all |
| after | No | Pagination cursor from previous response | |
| limit | No | Number of results (1-25) | |
| query | Yes | Search query | |
| timeout | No | Request timeout in seconds (default: 10) | |
| subreddit | No | Limit to subreddit (without r/) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states that the tool returns metadata and URLs, implying it is read-only, but does not disclose rate limits, authorization needs, pagination behavior (despite the 'after' parameter), or error handling. The behavioral description is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences, front-loaded with the primary purpose, and includes a complementary instruction to use 'fetch' for full content. Every word is necessary and there is no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 7 parameters and no output schema. The description is too brief to be complete: it omits explanation of the return structure beyond 'metadata and URLs', does not explain pagination (despite the 'after' parameter), and does not differentiate from the sibling 'browse_reddit' tool. More context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the input schema already documents all 7 parameters. The description adds no additional meaning beyond what is in the schema (e.g., it does not explain the purpose of 'sort', 'time', or 'after'). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches Reddit posts and returns metadata and URLs. It distinguishes from siblings like 'browse_reddit' by indicating that this tool is for searching, not browsing, and that full content requires another tool. However, it does not explicitly differentiate from the generic 'search' tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a usage hint: use another tool ('fetch') for full content. However, it does not give explicit context on when to use this over siblings like 'browse_reddit' or 'search', nor does it mention prerequisites or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v3.3.1browse_redditfetchget_alibaba_productget_aliexpress_productget_linkedin_jobsearchsearch_alibabasearch_aliexpresssearch_linkedin_jobssearch_marketplacesearch_realtorsearch_redditScored across 12 tools
Each tool targets a specific platform or action (search vs. get details), with clear boundaries. The only potential confusion is between the generic 'search' and platform-specific searches, but they are distinct in purpose (web search vs. platform search).
Most tools follow a 'verb_platform' pattern (e.g., search_aliexpress, get_alibaba_product). There are minor deviations like 'browse_reddit' and 'fetch' that break the pattern, but overall it is consistent.
With 12 tools covering multiple domains (e-commerce, social media, jobs, real estate, web), the count is well-scoped. Each tool serves a distinct purpose, and the number is not excessive.
The server provides search and detail retrieval for most platforms. Notable gaps include no dedicated detail tool for marketplace listings (Kijiji, etc.) beyond fetch, but the recommended workflow using fetch covers it. Overall, the surface is adequate for information retrieval.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Web search, browser automation, scraping, crawling and CAPTCHA solving for AI agents.
Direct access to 60+ scraping and search tools. Extract structured data from Google (Search, Maps, Trends), Amazon, Airbnb, Social Media, and any web page directly into your AI agent.
Reliable web fetching for AI agents with retry, circuit breaker, caching, and anti-bot bypass
Send quick greetings, scrape website content, and generate text or images on demand. Perform web s…
