Skip to main content
Glama

Promas (Product Image Scraper)

Promas is an automated product image scraper and Model Context Protocol (MCP) server for AI agents.

Instead of writing fragile per-site scrapers that break whenever HTML structures change, Promas combines Pluggable Search-Driven Discovery with a Universal Semantic Extraction Pipeline and Extensible CDN-Aware Upscaling. It reliably extracts master-resolution product photography across any brand or retail site (Apple, Nike, Sony, Amazon, Target, B&H, Best Buy, eBay, Shopify stores, and arbitrary product URLs).


1. Quickstart

pip install promas
playwright install chromium

Run instantly from anywhere:

promas "iPhone 16 Pro"

Option B: Run with Docker (Zero local Python / Playwright setup)

# Build the Docker image
docker build -t promas .

# Run the FastMCP Server for your AI Agent
docker run -i --rm promas

# Or run the standalone CLI scraper
docker run --rm promas promas "Sony FX3"

Related MCP server: Commerce MCP Server

2. Architecture & Pipeline

graph TD
    User([AI Agent / CLI Request]) --> Cache{TTL Disk Cache}
    Cache -- "Cache Hit (<0.1s)" --> Return([Return Verified Master Assets])
    Cache -- "Cache Miss" --> SearchRouter{Pluggable Discovery}
    SearchRouter -->|BRAVE_API_KEY| BraveAPI[Brave Search API]
    SearchRouter -->|SERPAPI_API_KEY| SerpAPI[SerpAPI Google]
    SearchRouter -->|Default: Free / Zero Keys| BrowserSearch[Stealth Browser Discovery]
    BraveAPI --> Scorer[E-Commerce Candidate Scorer]
    SerpAPI --> Scorer
    BrowserSearch --> Scorer
    Scorer --> ParallelScraper[Parallel Multi-Page Scraper]
    subgraph Scraping Pipeline
        ParallelScraper --> RateLimiter[Domain Rate Limiter]
        RateLimiter --> Parser[Universal Extractor: Schema.org, OG, Microdata, DOM]
        Parser --> CDNUpscaler[CDN Master Upscalers: Scene7, Nike, Shopify, Amazon...]
    end
    CDNUpscaler --> Verifier[Async HTTP Verification & pHash Dedup]
    Verifier --> CacheStore[(Save to Disk Cache)]
    CacheStore --> Return

3. Why Promas? (Comparison)

Feature

Raw Scripts (BeautifulSoup, Puppeteer)

Paid APIs (Bright Data, ScrapingBee)

Promas

Cost

Free

$50–$500+/mo recurring

100% Free & Open-Source (MIT)

Setup & Maintenance

Fragile per-site selectors; breaks on redesigns

Generic HTML responses; requires custom parsers

Search-Driven + Semantic Schemas + CDN Upscalers

Anti-Bot & Rate Limits

Blocked quickly by Cloudflare/Akamai

Handled in cloud

Per-Domain Rate Limiter + Stealth + Tenacity Retries

Image Verification

None; returns broken links & 1x1 pixels

Basic status check

Async HTTP validation + pHash perceptual dedup

Search Backends

Hardcoded scrapers

Custom API scrapers

Pluggable (Brave API / SerpAPI / Browser Fallback)

Caching

None

Extra cost

Built-in TTL Disk Cache (Sub-second repeated queries)

Image Quality

Usually captures low-res UI thumbnails

Raw page images only

Master CDN de-capping (up to 2500px+)

AI Agent Native

Manual wrapper needed

REST API only

Native FastMCP Tool Protocol + Docker support


4. Search Backends & Configuration

Promas works 100% out of the box with zero configuration or API keys required.

How Search Discovery Works:

  1. Free / Default (Zero Setup): Promas uses its built-in Playwright Stealth browser engine to discover e-commerce candidates via Bing and DuckDuckGo for free.

  2. Brave Search API (Recommended for Production): If BRAVE_API_KEY is set, Promas switches to official, ToS-compliant, sub-second API discovery.

  3. SerpAPI (Alternative): If SERPAPI_API_KEY is set, Promas queries Google Search via SerpAPI.

    • Get a free key (100 free queries/month): SerpAPI

Environment Variables Reference:

Variable

Default

Description

BRAVE_API_KEY

None

Optional API key for Brave Search

SERPAPI_API_KEY

None

Optional API key for SerpAPI Google Search

PROMAS_GLOBAL_CONCURRENCY

3

Max simultaneous browser contexts

PROMAS_PER_DOMAIN_CONCURRENCY

1

Max concurrent requests per target store

PROMAS_DOMAIN_DELAY_SECONDS

0.5

Polite delay between requests to the same domain

PROMAS_CACHE_ENABLED

True

Toggle disk caching (True/False)

PROMAS_CACHE_TTL_SECONDS

86400 (24h)

Cache expiration time in seconds

PROMAS_ENABLE_IMAGE_VERIFICATION

True

Async HTTP MIME & pixel dimension check

PROMAS_ENABLE_PERCEPTUAL_DEDUP

True

pHash near-duplicate crop removal

PROMAS_PHASH_HAMMING_THRESHOLD

4

Sensitivity threshold for pHash deduplication


5. Usage

A. Standalone CLI

Search by Product Name:

promas "iPhone 16 Pro"

Extract from a Direct URL:

promas "https://www.apple.com/iphone-16-pro/"

Filter by Specific Domain & Limit Count:

promas "Sony FX3" --max-images 5 --site bhphotovideo.com

Bypass Cache or Disable Verification:

promas "Nike Air Jordan 1" --no-cache --no-verify

B. FastMCP Server

Run the standalone MCP server:

promas-mcp

C. Agent Integration (mcp_config.json)

Native Python Installation:

{
  "mcpServers": {
    "promas": {
      "command": "promas-mcp",
      "env": {
        "BRAVE_API_KEY": "optional-key-here"
      }
    }
  }
}

Docker Container (Zero-dependency setup):

{
  "mcpServers": {
    "promas": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "promas"]
    }
  }
}

6. Multi-Tool MCP Suite

Promas exposes dedicated granular tools to AI agents:

Tool

Purpose

Description

fetch_product_images

Full Pipeline

Query string or URL -> Automated discovery -> Parallel scrape -> HTTP validation -> pHash dedup -> Master photo links.

search_product_urls

Discovery Only

Query string -> Fast ranking of candidate e-commerce product pages (returns URLs without scraping images).

scrape_single_url

Extraction Only

Specific direct URL -> Extracts title and master image assets strictly from that page.


7. Agent System Prompt Guidelines

Add the following instructions to your AI agent's system prompt:

You have access to the `fetch_product_images` tool (Promas), which universally retrieves high-resolution product imagery and official product links from across the web.

GUIDELINES FOR USING PROMAS:
1. When asked for product images, photos, or visual references, call `fetch_product_images(query=<specific product name or direct URL>)`.
2. Do not invent or hallucinate image URLs; strictly return the verified URLs returned by Promas.
3. Render primary high-res images as markdown (e.g. `![Product Photo](url)`) and provide source product links for reference.
4. If a specific store is desired by the user, you can supply `site_filter` (e.g. `site_filter="apple.com"`).

8. Integration Examples & Frameworks

Promas includes copy-pasteable configuration snippets and runnable example scripts in the examples/ directory:

  • 🤖 Claude Desktop: Complete setup guide and claude_desktop_config.json.

  • 💻 Cursor IDE: mcp.json config and prompt examples for Cursor Composer.

  • 🦜🔗 LangChain: Structured @tool wrapper and ReAct agent script.

  • ⚡ OpenAI API: Native OpenAI Tool / Function Calling implementation.

  • 📦 Smithery: One-click installation via npx -y @smithery/cli install promas --client claude.


9. Output Schema

{
  "status": "success",
  "query": "iPhone 16 Pro",
  "title": "Apple iPhone 16 Pro",
  "sources_scraped": [
    "https://www.target.com/p/apple-iphone-16-pro/-/A-93597960",
    "https://www.amazon.com/Apple-iPhone-Version-256GB-Titanium/dp/B0DHJDPYYR",
    "https://www.apple.com/shop/buy-iphone/iphone-16"
  ],
  "images": [
    "https://target.scene7.com/is/image/Target/GUEST_7c0750b4-ee18-41d4-9309-d08e41619229",
    "https://target.scene7.com/is/image/Target/GUEST_4e1ce623-313f-4193-93a4-61dc0fc9da14"
  ],
  "error_message": null
}

10. Contributing

Contributions are warmly welcomed! Adding master upscaling support for a new e-commerce platform or CDN takes less than 10 lines of code with our decorator plugin registry.

See CONTRIBUTING.md for step-by-step instructions on adding a new CDN rule.


11. Testing & Quality Assurance

Promas includes unit tests for pure parsing functions, type checks, and canary integration tests:

# Run unit tests
pytest -v

# Run linting
ruff check .

# Run type checker
mypy promas/ tests/

# Run live golden canary integration tests (hits live sites)
pytest -v --run-integration

  • Public Access Only: Promas accesses exclusively publicly available web pages; it does not bypass authentication, paywalls, or private logins.

  • Terms of Service: Automated access may be subject to individual site Terms of Service. Always review target domains' ToS and robot policies before scraping at scale, or supply official search API keys (BRAVE_API_KEY / SERPAPI_API_KEY) for ToS-compliant discovery.

  • Image Copyright & Attribution: Promas resolves and returns direct image URLs — it does not store, rehost, or copy media files. Downstream display, storage, or commercial use of retrieved imagery is the user's responsibility.


13. License

This project is licensed under the MIT License.

Available Tools

4 tools
fetch_product_imagesA

Universally extracts verified, high-resolution product photography and official sources across the web.

USE THIS TOOL WHEN:

  • You need product photos, packshots, visual references, or master CDN assets for any physical item.

  • You want Promas to automatically discover authoritative stores, scrape candidate pages, verify images via HTTP, and deduplicate near-identical photo crops.

EXAMPLE QUERIES:

RETURNS:

  • ProductImageResult containing:

    • status: 'success' or 'error'

    • title: Cleaned product title

    • sources_scraped: List of retail & brand URLs visited

    • images: List of verified master-resolution image URLs (up to 2500px+)

Inputs: query (required), no_cache, max_images, site_filter. Returns: Unified result model returned by Promas.

get_product_images_toolD

Backward-compatible alias for fetch_product_images.

Inputs: query (required), max_images, site_filter. Returns: Unified result model returned by Promas.

scrape_single_urlA

Extracts high-resolution product imagery from a single direct product URL (Extraction only).

USE THIS TOOL WHEN:

  • You already have an exact product URL (from search_product_urls or user input) and want to extract master photos strictly from that page.

  • Uses universal Schema.org JSON-LD, OpenGraph, dynamic JS datasets, DOM selectors, and CDN upscalers.

EXAMPLE CALL:

RETURNS:

  • ProductImageResult containing extracted images and metadata from the specified URL.

Inputs: url (required), no_cache, max_images. Returns: Unified result model returned by Promas.

search_product_urlsA

Discovers and ranks candidate e-commerce product pages for a given query (Discovery only).

USE THIS TOOL WHEN:

  • You want to find authoritative product detail pages (PDPs) across retailers without downloading or scraping images yet.

  • You want to preview which stores/URLs exist for a product before selecting one to scrape.

EXAMPLE CALL:

  • search_product_urls(query="Sony FX3 Cinema Camera", max_urls=3)

  • search_product_urls(query="MacBook Pro M3", site_filter="apple.com")

RETURNS:

  • A list of ranked, direct product destination URLs.

Inputs: query (required), max_urls, site_filter. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.4/5.0

    Scored across 4 tools

    Disambiguation5/5

    The three primary tools have clearly distinct purposes: fetch_product_images performs full discovery and scraping, search_product_urls only finds URLs, and scrape_single_url extracts images from a provided URL. The fourth tool is explicitly marked as a backward-compatible alias, so no genuine ambiguity exists.

    Naming Consistency4/5

    The main tools follow a verb_noun pattern (fetch_product_images, search_product_urls, scrape_single_url), but the alias get_product_images_tool breaks the pattern by using 'get' and a '_tool' suffix. This is a minor deviation, as the alias is clearly labeled and does not detract from the overall predictability.

    Tool Count5/5

    With only 4 tools, the server is well-scoped. Each core tool serves a distinct stage in the product image retrieval workflow (discovery, extraction, combined pipeline), and the alias is a harmless addition. The count fits comfortably within the ideal range.

    Completeness5/5

    The tool surface completely covers the domain: it offers discovery-only (search_product_urls), extraction-only (scrape_single_url), and a comprehensive pipeline (fetch_product_images). There are no obvious dead ends or missing operations for the stated purpose of retrieving product images.

    Related MCP Connectors

    Related MCP Servers