scrapewise_get_scraper_data_group
Returns the scraped product rows from a group's MongoDB collection, paginated.
Each row is one scraped product (or row variant per scraper config). Filters
can narrow the result to specific custom categories or to the latest run only.
This is the portal-style endpoint (consumed by the customer's UI). For the
SDK / MCP-friendly variant, use scrapewise_get_scraper_data_group_client.
Args:
- id (string, path param, required): the group's MongoDB ObjectId.
- filters (string, query param, optional): URL-encoded JSON filter expression.
Pass an empty string for no filtering. One entry per column. Say which
operator you mean with an envelope:
{"_sw_endpoint": {"$op": "equals", "$type": "text", "$value": "amazon"}}
$op is one of equals / notEquals / contains / notContains / startsWith /
endsWith / blank / notBlank / greaterThan / greaterThanOrEqual / lessThan /
lessThanOrEqual / inRange / in / notIn. $type is the column family
(text / number / date / boolean / set) and decides how the value is compared
— an equals on a date column is an exact date, never a text match. Text
matching is CASE-INSENSITIVE and the value is matched literally, so
equals "amazon" selects "amazon" and NOT "amazonseller".
A bare value with no envelope means CONTAINS, which is what older clients
send. A filter that cannot be read is answered HTTP 400 INVALID_FILTER
naming the column — never a silent full table.
- categories (List<string>, query param, optional): customer-defined category
names to restrict the result. Multiple ?categories=foo&categories=bar.
- isLastRun (boolean, query param, optional): if true, only includes rows from
the most recent completed scraper-job-status across the group.
- pad (boolean, query param, optional, default TRUE): every row carries every
column of the group's union, with an explicit null for any column that row
lacks. Pass `pad=false` for the slim shape: rows carry only the keys they
actually have, and the envelope returns `hasMore` + `nextCursor` with the
total counted once on the first page instead of `totalPages` /
`totalElements` / `last`. Padding is the default and is removed entirely
only in a later, announced release - it is safe to ignore this parameter.
- ctx (boolean, query param, optional, default false): also return the
per-call context documents as a `contexts` sidecar, keyed by each row's
`_sw_result_set_id`. A context (the query, the store, the result counts,
the paging tokens) is stored ONCE per upstream call, not copied onto every
row, so ask for it only when you intend to read a `ctx_*` column. The
sidecar is byte-capped per page; `contextsTruncated: true` means only the
first result sets in page order are included.
- cursor / cursorId / cursorSort (query params, optional): keyset paging for the slim
shape. Echo back the `nextCursor` object from the previous page. Ignored
while padding is on, where pages are addressed by `page` as before.
- pageable (Spring Pageable, query params): page, size, sort. Default size 100.
Returns:
Spring Page<Document> — `{content, totalElements, totalPages, ...}` where
each content item is a row (Map of field-name → value as written by the
scrapers). Field shape is dynamic and depends on the scraper's itemsConfig.
Rows also carry the reserved `_sw_*` bookkeeping columns - which endpoint
wrote the row, whether it worked, what it cost and what was sent - including
`_sw_status` (OK / NO_RESULTS / INVALID_INPUT / UPSTREAM_ERROR / PARSE_FAILED),
`_sw_error_message`, `_sw_source_url` and `_sw_run_date`. An input that failed
still produces a row; `_sw_status` is how you tell it from data.
`_sw_scraper` names the scraper that wrote the row (the customer's own name
for it, e.g. "22 Google news") and `_sw_scraper_id` is its id - which is how
a row is attributed in a group that mixes many scrapers. Both are filterable
and sortable like any column; a row written before the name was stamped has
it resolved on the read from its run.
Every row also carries `_sw_row_id`: that row's own id as a string, on every
page shape (padded and `pad=false`, offset and keyset, `isLastRun` or not).
It is what `scrapewise_run_scraper_data_group_hotels_detail` takes in its
`rowIds`. It is added by this read and stored nowhere, so it is not part of
the group's column union and is absent from the Excel export.
With `pad=false` the envelope is `{content, hasMore, nextCursor, totalCount}`.
With `ctx=1` the response also carries `contexts` and `contextsTruncated`.
Examples:
- Use when: "show me page 1 of group {id}'s products" → id, page=0, size=20
- Use when: "filter group {id} to category 'shoes'" → categories=shoes
- Use when: "only show last-run results" → isLastRun=true
- Don't use when: building agent tooling → use scrapewise_get_scraper_data_group_client
(offers ?sanitized=true for prompt-injection safety)
- Don't use when: you want CSV export → use scrapewise_export_scraper_data_group
Error Handling:
- 400 (CustomerError envelope) if no group with the given id exists for this customer.
- 401 Unauthorized.
- 500 INTERNAL.[runtime hint] This tool returns JSON-as-text — its response shape is dynamic (per-scraper itemsConfig, file binary, etc.) so the MCP outputSchema is omitted. Do not rely on the presence of any specific field in the response; treat the result as a string-or-object blob and parse defensively.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"id": "<id>"
},
"name": "scrapewise_get_scraper_data_group"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | ||
| ctx | No | Also return the per-call `contexts` sidecar, keyed by each row's `_sw_result_set_id`. | |
| pad | No | Return every column of the group union on every row (default). Pass false for the slim shape. | |
| cols | No | Portal only. `runs`: pad rows to the columns of the latest run of each scraper (plus the page's own keys). | |
| page | No | Zero-based page index (0..N) | |
| size | No | The size of the page to be returned | |
| sort | No | Sorting criteria in the format: property,(asc|desc). Default sort order is ascending. Multiple sort criteria are supported. | |
| total | No | Portal only. `fast`: estimated or capped total; the page then carries `totalIsLowerBound` when the total is a lower bound. | |
| cursor | No | Keyset cursor: the previous page's `nextCursor.key`. Echoed for readability; `cursorId` is authoritative. | |
| filters | No | ||
| cursorId | No | Keyset cursor: the previous page's `nextCursor.id`. Slim shape only. | |
| isLastRun | No | ||
| categories | No | ||
| cursorSort | No | Keyset cursor: the previous page's `nextCursor.sort`. Echo it back; a cursor replayed against a different sort is rejected with 400. |