ai_scraper
Scrape ChatGPT or Gemini answers for a keyword to extract cited sources and the ranked local businesses each AI recommends. Use it to track where a business appears in AI search results.
Instructions
Use when scraping ChatGPT or Gemini search results for a keyword. Returns markdown response, local_businesses, sources, fan_out_queries, and brand_entities. Each entry in sources carries a position — the order it was cited, from 1, after duplicates collapse. Costs 4 credits.
TWO RANKINGS, DO NOT MERGE THEM: local_businesses is the platform's LISTING, in the order it emitted cards. recommended_businesses is the SHORTLIST read out of the answer's own prose, in the order the answer argued for it, with shortlist_rank, listed, mention, is_top_pick and the citation offered beside each entry. listed: true means the entry held a place in a list the answer presented and shortlist_rank is that place; listed: false means the answer named it in a sentence instead, so shortlist_rank is null rather than a position it never had. mention is the answer's own words about the business, plain text — the evidence behind the row. They disagree routinely — ChatGPT opens a local answer by dumping its whole result set and only then writes the list it recommends, so a business the prose puts third can sit tenth in the array. Quote shortlist_rank when a client asks where an AI RECOMMENDS them.
GEMINI HAS NO LISTING AT ALL: it sends no business block, so local_businesses is empty on every gemini local query and recommended_businesses is the only place a business appears. An empty array on gemini is not evidence that nobody was named.
LOCAL BUSINESSES: on a local question, chat_gpt lists businesses inside its answer and we return them as local_businesses — rank, name, category, rating, reviews_count, address, phone, website, domain. Field names it shares with /v1/serp/local-pack mean the same things, so code reading those works against both — but the sets differ: this adds category and domain, and has no cid or hours. gemini has no equivalent and returns an empty array — read recommended_businesses there.
FORCE A SEARCH ON CHAT_GPT: chat_gpt sometimes answers a local prompt with a clarifying question ("which Orange County do you mean?") instead of an answer, which reads as an empty result. force_web_search: true makes it answer, at the same 4 credits, and is what makes the local_businesses block appear reliably. gemini decides for itself and returns a 400 if you ask.
MATCHING CITATIONS ON GEMINI: local answers often cite Google Business Profiles, which arrive with domain "maps.google.com" and the business name in title, identified by the cid in the URL. Match on title or cid rather than domain for those.
search_results is a reserved field we have not observed populated on either platform — read sources. fan_out_queries and brand_entities depend on the query and platform and are often empty.
SLOW: we ask ChatGPT or Gemini the question live and wait for the full answer. May take 30-90 seconds. This is normal, not an error — do not retry a call still in flight, it starts a second billable job.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| fresh | No | Bypass the hour of caching and run the query again. Use when sampling the same prompt repeatedly to measure how the answer varies — a cached hit would return the earlier run and hide the variance. Always billed at the normal price, since there is no cache hit to be free. | |
| keyword | Yes | Keyword to scrape results for (e.g. "best electrician") | |
| language | No | Language to ask in, as ISO 639-1 code or name (e.g. "fr", "French"). Defaults to the language of the market you pass — a French location is asked in French. This tool queries the model live and covers 43 languages worldwide, so this is not limited the way the AI mention endpoints are. | |
| location | No | Location for results (e.g. "Boston, MA"). Gemini supports city-level; ChatGPT resolves to country. Default: US | |
| platform | Yes | Which platform to scrape (chat_gpt or gemini) | |
| force_web_search | No | Require the platform to search the web before answering. chat_gpt only; gemini returns a 400. Does not change the price. Reach for it when a local prompt comes back as a clarifying question rather than an answer. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| keyword | Yes | ||
| sources | Yes | ||
| markdown | Yes | ||
| platform | Yes | ||
| brand_entities | Yes | ||
| reader_version | No | ||
| search_results | Yes | ||
| fan_out_queries | Yes | ||
| local_businesses | No | ||
| recommended_businesses | No |