MCP Price Scanner

A small, self-hosted MCP server that lets an AI agent read public marketplace listings.

Agents running on cloud VMs are routinely blocked by marketplace bot defenses (DataDome, Cloudflare, login walls) because of their datacenter IP addresses and sandbox limits. This bridge runs on your own home server, on a residential connection, fetches pages with a real browser (Playwright + Chromium) and returns structured listings over MCP. The agent calls the tools; the bridge fetches and parses.

The full design is in SPEC.md.

Read-only boundary

The bridge is a reader. It never buys, bids, makes offers, messages sellers, follows listings, or performs any account action, and the code contains no such feature. It does not get around blocks either: a 403, 429, challenge page or CAPTCHA wall is reported as blocked and the source is left alone for a cooldown period. No CAPTCHA solving, no user-agent rotation, no fingerprint tricks beyond a normal, current Chromium.

Your watch lists, criteria and alert rules are not part of this project. They live on the agent side and arrive as plain tool arguments.

Related MCP server: mcp-web-search

Tools

All tools return JSON. Every response carries a status: ok, blocked, login-required, disabled or error.

A blocked result looks like this; it contains no listings:

{"site": "allegro", "query": "road bike", "status": "blocked", "reason": "http_403",
 "retry_after_seconds": 3600, "listings": [], "count": 0}

Listing schema

{
  "id": "source-native listing id or URL slug",
  "site": "olx",
  "title": "listing title, verbatim",
  "price": 1234.56,
  "currency": "PLN",
  "location": "city or branch, as shown",
  "url": "https://www.example-market.test/item/123",
  "posted_date": "2026-10-24T10:16:00+02:00",
  "date_reliable": true,
  "condition": "as shown, else null",
  "snippet": "short description excerpt, else null",
  "fetched_at": "2026-10-24T20:50:00+00:00",
  "flags": []
}

• Nothing is invented. A field the page does not show is null.

• posted_date is an ISO date, or an ISO date-time with offset when the page shows a time. If the page shows no date, posted_date is null and date_reliable is false. Dates shown as an age ("2 days ago") and dates from sources whose dates are known to be unreliable are returned with date_reliable: false. A "refreshed on" date is not a posting date and is returned as null.

• price is the number shown and currency the currency shown. No conversion, no fees, no shipping estimates.

• flags lists markings the listing itself shows, e.g. "replica". They are surfaced, never hidden. (This field is an addition to the schema in the spec.)

Shipped sources

Configured in config/sources.example.yaml. Enable only the ones you need.

Install

Target: a small Debian-stable LXC container (or any Linux host). Suggested size 2 vCPU, 4 GB RAM, 16 GB disk. Python 3.12 or newer.

# 1. System packages and a service user (as root)
apt-get update && apt-get install -y python3 python3-venv git curl openssl
useradd --system --home-dir /opt/price-scanner --create-home --shell /usr/sbin/nologin price-scanner

# 2. Code and virtualenv (as the service user)
sudo -u price-scanner -H bash -c '
  cd /opt/price-scanner &&
  git clone https://github.com/tuga-works/mcp-price-scanner.git &&
  python3 -m venv venv &&
  venv/bin/pip install ./mcp-price-scanner &&
  venv/bin/playwright install chromium'

# 3. Chromium system libraries (as root)
/opt/price-scanner/venv/bin/playwright install-deps chromium

sudo is used for brevity; runuser -u price-scanner -- ... works the same where sudo is not installed.

Configure

Only two files are yours, and git ignores both: .env and config/sources.yaml.

cd /opt/price-scanner/mcp-price-scanner
# The token: generated straight into the file, never echoed.
umask 077
{ grep -v '^PRICE_SCANNER_TOKEN=' .env.example
  printf 'PRICE_SCANNER_TOKEN=%s\n' "$(openssl rand -hex 32)"; } > .env
chown price-scanner:price-scanner .env

# Sources: copy the example and enable what you need.
install -m 640 -o price-scanner -g price-scanner config/sources.example.yaml config/sources.yaml

Environment variables (see .env.example):

Adding a source

A plain HTML source needs no code: add an entry to config/sources.yaml with a search URL pattern, the CSS selector of one result card, and a selector per field. The header of config/sources.example.yaml documents every option. A minimal example:

sources:
  example-shop:
    name: Example shop
    enabled: true
    base_url: https://shop.example/
    default_currency: EUR
    search:
      url: https://shop.example/search?q={query_plus}
      sort: {newest: sort=new, price_asc: sort=price}
      item: '.product'
      fields:
        title: 'h2'
        url: {selector: 'a', attr: href}
        price: '.price'
        location: {selector: '.meta', label: 'Location:'}

A site that needs custom parsing gets a small named module in price_scanner/parsers/ and parser: <name> in its config entry (facebook-marketplace is the example).

Run

sudo cp systemd/price-scanner.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now price-scanner
journalctl -u price-scanner -f

The unit runs as the non-root price-scanner user, restarts on failure and logs to the journal. For a manual foreground run: venv/bin/price-scanner from the repository directory (it reads .env there).

Check it

From the bridge host, with the token taken from .env without printing it:

cd /opt/price-scanner/mcp-price-scanner
TOKEN=$(sed -n 's/^PRICE_SCANNER_TOKEN=//p' .env)
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://127.0.0.1:8765/mcp   # 401: no token
curl -s -X POST http://127.0.0.1:8765/mcp \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"check","version":"0"}}}'

Or with the Python client that ships with FastMCP:

import asyncio, os
from fastmcp import Client

async def main():
    async with Client("http://127.0.0.1:8765/mcp", auth=os.environ["TOKEN"]) as c:
        print((await c.call_tool("health", {"probe": True})).data)

asyncio.run(main())

Expose it to your agent

The server listens on localhost. Expose it over your own private network, for example a WireGuard/Tailscale-style tailnet or a reverse proxy, never directly to the internet. Then point the agent's MCP client at http://<your-host>.<your-tailnet>:<port>/mcp (or your proxy's URL) with the header Authorization: Bearer <your-token>, and make one health() call before any search. If a proxy forwards a different Host header, add it to PRICE_SCANNER_ALLOWED_HOSTS.

Politeness rules

• One request at a time per site, across all tool calls.

• A random delay of 2–5 seconds (configurable per source, never below 1 s) between two requests to the same site.

• On HTTP 429 or 403, a challenge page or a CAPTCHA wall: stop, report blocked, and send nothing to that source for the cooldown (default one hour). Calls during the cooldown return blocked with retry_after_seconds immediately.

• One search call fetches one results page. No crawling, no pagination loops.

• Chromium's default user agent; no fake identities. Images, media and fonts are not downloaded, which saves the sites bandwidth. The optional plain-HTTP mode (fetch_mode: http) uses curl_cffi presenting as a current Chrome.

• Pages are cached in memory for a few minutes to avoid duplicate fetches within one agent run. Nothing is stored on disk.

Login-gated sources

Login-gated sources (facebook-marketplace) stay disabled unless you enable them. To enable one:

1. Create a dedicated account for automated reads. Never use a personal main account.

2. On the bridge host, create a Playwright persistent profile outside the repository and log in to it by hand (this needs a display, e.g. a temporary desktop session or X forwarding): venv/bin/python -c "from playwright.sync_api import sync_playwright as p; b=p().start().chromium.launch_persistent_context('<profile-dir>', headless=False); b.pages[0].goto('https://www.facebook.com/'); input('Log in, then press Enter'); b.close()"

3. Set PRICE_SCANNER_PROFILE_DIR=<profile-dir> in .env, set enabled: true for the source in config/sources.yaml, restart.

The project stores no credentials. The profile directory is yours, outside the repository, and listed in .gitignore patterns in case it is ever created inside it.

Keys and secrets

• No secret belongs in this repository: no tokens, passwords, cookies, session files, browser profiles, API keys or private keys.

• Secrets come only from the environment (PRICE_SCANNER_TOKEN, PRICE_SCANNER_PROFILE_DIR). .env.example holds placeholders only.

• Token comparison is constant-time. A missing or wrong token gets a 401 with an empty body.

• Logs pass through a redaction filter that removes the token, the profile path and any Authorization, Cookie or Set-Cookie values.

SSRF protection

fetch_listing takes URLs from the caller, and the bridge sits on a home network, so it must never become a way to reach other machines:

• A listing URL is accepted only in a plain form: http(s) scheme, a host name (no IP literals, including numeric forms such as 2130706433), default port, and no backslash, @, credentials, whitespace or control characters. The URL is rebuilt from its parsed parts (fragment dropped, http upgraded to https) and only the rebuilt string is fetched. Its host must equal, or be a subdomain of, a configured source's domain.

• All browser and HTTP traffic, including redirect hops and page subresources, leaves through a built-in egress guard: a loopback-only CONNECT proxy that tunnels TLS on port 443 to public addresses only. It resolves each host itself, refuses loopback, private, link-local, carrier-grade NAT/overlay, multicast and reserved addresses, and connects to the address it checked. TLS stays end to end, so the browser's identity is unchanged.

• Document navigations (pages and frames) outside the source's domains are aborted, and a page whose final URL left the source's domains is discarded without being parsed or cached.

For defence in depth, also restrict the container's outbound traffic to the internet at the network level if your setup allows it.

Development

python3 -m venv .venv && .venv/bin/pip install -e '.[dev]'
.venv/bin/playwright install chromium
.venv/bin/pytest

The tests use synthetic HTML fixtures only and send no network requests.

Release checklist

Before any public release, scan the full history for secrets and record the result (not the secrets) in the release notes:

gitleaks git --redact -v .

Limitations (v1)

• First results page only; limit caps at the source's max_results.

• Site markup changes break selectors; an ok result with count: 0 and a note usually means the layout changed.

• Allegro selectors could not be verified while the site was blocking.

• No price history, alerting, notifications or web UI. The calling agent owns all of that.

License

Not chosen yet; see LICENSE.