Skip to main content
Glama

Promas(产品图片抓取器)

Promas 是一个面向 AI 代理的自动化产品图片抓取器和 Model Context Protocol (MCP) 服务器。

无需编写脆弱的逐站点抓取器(一旦 HTML 结构变化就会失效),Promas 将可插拔的搜索驱动发现与通用语义提取流水线及可扩展的 CDN 感知放大相结合。它能够在任何品牌或零售网站(Apple、Nike、Sony、Amazon、Target、B&H、Best Buy、eBay、Shopify 商店以及任意产品 URL)上可靠地提取主分辨率产品图片。


1. 快速开始

选项 A:从 PyPI 安装(推荐)

pip install promas
playwright install chromium

在任何地方即时运行:

promas "iPhone 16 Pro"

选项 B:使用 Docker 运行(无需在本地安装 Python / Playwright)

# Build the Docker image
docker build -t promas .

# Run the FastMCP Server for your AI Agent
docker run -i --rm promas

# Or run the standalone CLI scraper
docker run --rm promas promas "Sony FX3"

Related MCP server: Commerce MCP Server

2. 架构与流水线

graph TD
    User([AI Agent / CLI Request]) --> Cache{TTL Disk Cache}
    Cache -- "Cache Hit (<0.1s)" --> Return([Return Verified Master Assets])
    Cache -- "Cache Miss" --> SearchRouter{Pluggable Discovery}
    SearchRouter -->|BRAVE_API_KEY| BraveAPI[Brave Search API]
    SearchRouter -->|SERPAPI_API_KEY| SerpAPI[SerpAPI Google]
    SearchRouter -->|Default: Free / Zero Keys| BrowserSearch[Stealth Browser Discovery]
    BraveAPI --> Scorer[E-Commerce Candidate Scorer]
    SerpAPI --> Scorer
    BrowserSearch --> Scorer
    Scorer --> ParallelScraper[Parallel Multi-Page Scraper]
    subgraph Scraping Pipeline
        ParallelScraper --> RateLimiter[Domain Rate Limiter]
        RateLimiter --> Parser[Universal Extractor: Schema.org, OG, Microdata, DOM]
        Parser --> CDNUpscaler[CDN Master Upscalers: Scene7, Nike, Shopify, Amazon...]
    end
    CDNUpscaler --> Verifier[Async HTTP Verification & pHash Dedup]
    Verifier --> CacheStore[(Save to Disk Cache)]
    CacheStore --> Return

3. 为什么选择 Promas?(对比)

功能

原始脚本(BeautifulSoup、Puppeteer)

付费 API(Bright Data、ScrapingBee)

Promas

成本

免费

每月 50–500+ 美元订阅费

100% 免费且开源(MIT)

设置与维护

脆弱的逐站点选择器;改版即失效

通用 HTML 响应;需自定义解析器

搜索驱动 + 语义模式 + CDN 放大器

反爬虫与速率限制

很快被 Cloudflare/Akamai 拦截

云端处理

按域速率限制器 + Stealth + Tenacity 重试

图片验证

无;返回失效链接和 1x1 像素

基本状态检查

异步 HTTP 验证 + pHash 感知去重

搜索后端

硬编码抓取器

自定义 API 抓取器

可插拔(Brave API / SerpAPI / 浏览器回退)

缓存

无

额外费用

内置 TTL 磁盘缓存(重复查询亚秒级响应)

图片质量

通常捕获低分辨率 UI 缩略图

仅返回页面原始图片

主 CDN 去上限(最高 2500px+)

AI 代理原生支持

需要手动封装

仅 REST API

原生 FastMCP 工具协议 + Docker 支持


4. 搜索后端与配置

Promas 100% 开箱即用,无需任何配置或 API 密钥。

搜索发现的工作原理:

  1. 免费 / 默认(零设置):Promas 使用内置的 Playwright Stealth 浏览器引擎,通过 Bing 和 DuckDuckGo 免费发现电子商务候选页面。

  2. Brave Search API(生产环境推荐):如果设置了 BRAVE_API_KEY,Promas 将切换到官方、符合 ToS、亚秒级响应的 API 发现方式。

  3. SerpAPI(替代方案):如果设置了 SERPAPI_API_KEY,Promas 将通过 SerpAPI 查询 Google 搜索。

    • 获取免费密钥(每月 100 次免费查询): SerpAPI

环境变量参考:

变量

默认值

描述

BRAVE_API_KEY

None

用于 Brave Search 的可选 API 密钥

SERPAPI_API_KEY

None

用于 SerpAPI Google 搜索的可选 API 密钥

PROMAS_GLOBAL_CONCURRENCY

3

最大并行浏览器上下文数

PROMAS_PER_DOMAIN_CONCURRENCY

1

每个目标商店的最大并发请求数

PROMAS_DOMAIN_DELAY_SECONDS

0.5

对同一域名发起请求之间的礼貌延迟(秒)

PROMAS_CACHE_ENABLED

True

切换磁盘缓存(True/False)

PROMAS_CACHE_TTL_SECONDS

86400 (24h)

缓存过期时间(秒)

PROMAS_ENABLE_IMAGE_VERIFICATION

True

异步 HTTP MIME 类型与像素尺寸检查

PROMAS_ENABLE_PERCEPTUAL_DEDUP

True

pHash 近似重复裁剪移除

PROMAS_PHASH_HAMMING_THRESHOLD

4

pHash 去重的灵敏度阈值


5. 使用方法

A. 独立 CLI

按产品名称搜索:

promas "iPhone 16 Pro"

从直接 URL 提取:

promas "https://www.apple.com/iphone-16-pro/"

按特定域名过滤并限制数量:

promas "Sony FX3" --max-images 5 --site bhphotovideo.com

绕过缓存或禁用验证:

promas "Nike Air Jordan 1" --no-cache --no-verify

B. FastMCP 服务器

运行独立 MCP 服务器:

promas-mcp

C. 代理集成(mcp_config.json)

原生 Python 安装:

{
  "mcpServers": {
    "promas": {
      "command": "promas-mcp",
      "env": {
        "BRAVE_API_KEY": "optional-key-here"
      }
    }
  }
}

Docker 容器(零依赖设置):

{
  "mcpServers": {
    "promas": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "promas"]
    }
  }
}

6. 多工具 MCP 套件

Promas 向 AI 代理提供专用的细粒度工具:

工具

用途

描述

fetch_product_images

完整流水线

查询字符串或 URL -> 自动发现 -> 并行抓取 -> HTTP 验证 -> pHash 去重 -> 主图链接。

search_product_urls

仅发现

查询字符串 -> 对候选电子商务产品页面进行快速排序(返回 URL,不抓取图片)。

scrape_single_url

仅提取

特定直接 URL -> 严格从该页面提取标题和主图片资源。


7. 代理系统提示词指南

将以下说明添加到你的 AI 代理的系统提示词中:

You have access to the `fetch_product_images` tool (Promas), which universally retrieves high-resolution product imagery and official product links from across the web.

GUIDELINES FOR USING PROMAS:
1. When asked for product images, photos, or visual references, call `fetch_product_images(query=<specific product name or direct URL>)`.
2. Do not invent or hallucinate image URLs; strictly return the verified URLs returned by Promas.
3. Render primary high-res images as markdown (e.g. `![Product Photo](url)`) and provide source product links for reference.
4. If a specific store is desired by the user, you can supply `site_filter` (e.g. `site_filter="apple.com"`).

8. 输出模式

{
  "status": "success",
  "query": "iPhone 16 Pro",
  "title": "Apple iPhone 16 Pro",
  "sources_scraped": [
    "https://www.target.com/p/apple-iphone-16-pro/-/A-93597960",
    "https://www.amazon.com/Apple-iPhone-Version-256GB-Titanium/dp/B0DHJDPYYR",
    "https://www.apple.com/shop/buy-iphone/iphone-16"
  ],
  "images": [
    "https://target.scene7.com/is/image/Target/GUEST_7c0750b4-ee18-41d4-9309-d08e41619229",
    "https://target.scene7.com/is/image/Target/GUEST_4e1ce623-313f-4193-93a4-61dc0fc9da14"
  ],
  "error_message": null
}

9. 贡献

热忱欢迎贡献!借助我们的装饰器插件注册表,为新的电子商务平台或 CDN 添加主图放大支持只需不到 10 行代码。

👉 有关添加新 CDN 规则的分步说明,请参阅 CONTRIBUTING.md。


10. 测试与质量保证

Promas 包含纯解析函数的单元测试、类型检查以及金丝雀集成测试:

# Run unit tests
pytest -v

# Run linting
ruff check .

# Run type checker
mypy promas/ tests/

# Run live golden canary integration tests (hits live sites)
pytest -v --run-integration

11. 法律与道德使用

  • 仅限公开访问:Promas 仅访问公开可用的网页;它不会绕过身份验证、付费墙或私有登录。

  • 服务条款:自动访问可能受各网站服务条款的约束。在规模化抓取之前,请务必查看目标域名的 ToS 和爬虫策略,或提供官方搜索 API 密钥(BRAVE_API_KEY / SERPAPI_API_KEY)以进行符合 ToS 的发现。

  • 图片版权与署名:Promas 解析并返回直接的图片 URL——它不会存储、重新托管或复制媒体文件。对检索到的图片进行下游展示、存储或商业使用由用户自行负责。


12. 许可证

本项目采用 MIT 许可证 授权。

Available Tools

4 tools
fetch_product_imagesA

Universally extracts verified, high-resolution product photography and official sources across the web.

USE THIS TOOL WHEN:

  • You need product photos, packshots, visual references, or master CDN assets for any physical item.

  • You want Promas to automatically discover authoritative stores, scrape candidate pages, verify images via HTTP, and deduplicate near-identical photo crops.

EXAMPLE QUERIES:

RETURNS:

  • ProductImageResult containing:

    • status: 'success' or 'error'

    • title: Cleaned product title

    • sources_scraped: List of retail & brand URLs visited

    • images: List of verified master-resolution image URLs (up to 2500px+)

Inputs: query (required), no_cache, max_images, site_filter. Returns: Unified result model returned by Promas.

get_product_images_toolD

Backward-compatible alias for fetch_product_images.

Inputs: query (required), max_images, site_filter. Returns: Unified result model returned by Promas.

scrape_single_urlA

Extracts high-resolution product imagery from a single direct product URL (Extraction only).

USE THIS TOOL WHEN:

  • You already have an exact product URL (from search_product_urls or user input) and want to extract master photos strictly from that page.

  • Uses universal Schema.org JSON-LD, OpenGraph, dynamic JS datasets, DOM selectors, and CDN upscalers.

EXAMPLE CALL:

RETURNS:

  • ProductImageResult containing extracted images and metadata from the specified URL.

Inputs: url (required), no_cache, max_images. Returns: Unified result model returned by Promas.

search_product_urlsA

Discovers and ranks candidate e-commerce product pages for a given query (Discovery only).

USE THIS TOOL WHEN:

  • You want to find authoritative product detail pages (PDPs) across retailers without downloading or scraping images yet.

  • You want to preview which stores/URLs exist for a product before selecting one to scrape.

EXAMPLE CALL:

  • search_product_urls(query="Sony FX3 Cinema Camera", max_urls=3)

  • search_product_urls(query="MacBook Pro M3", site_filter="apple.com")

RETURNS:

  • A list of ranked, direct product destination URLs.

Inputs: query (required), max_urls, site_filter. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.4/5.0

    Scored across 4 tools

    Disambiguation5/5

    The three primary tools have clearly distinct purposes: fetch_product_images performs full discovery and scraping, search_product_urls only finds URLs, and scrape_single_url extracts images from a provided URL. The fourth tool is explicitly marked as a backward-compatible alias, so no genuine ambiguity exists.

    Naming Consistency4/5

    The main tools follow a verb_noun pattern (fetch_product_images, search_product_urls, scrape_single_url), but the alias get_product_images_tool breaks the pattern by using 'get' and a '_tool' suffix. This is a minor deviation, as the alias is clearly labeled and does not detract from the overall predictability.

    Tool Count5/5

    With only 4 tools, the server is well-scoped. Each core tool serves a distinct stage in the product image retrieval workflow (discovery, extraction, combined pipeline), and the alias is a harmless addition. The count fits comfortably within the ideal range.

    Completeness5/5

    The tool surface completely covers the domain: it offers discovery-only (search_product_urls), extraction-only (scrape_single_url), and a comprehensive pipeline (fetch_product_images). There are no obvious dead ends or missing operations for the stated purpose of retrieving product images.

    Maintenance

    ActivityStale
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers