Select values on a page
select_valuesExtract text or attributes from any web page using CSS or XPath selectors and get each value's location.
Instructions
Say what a CSS or XPath selector gives on a page, and where each value is.
html_or_url: an http(s) URL to fetch, or the HTML itself. selector: CSS, with ::text for an element's own text and ::attr(name) for an attribute ("span.price::text", "a::attr(href)"), or XPath ("//h1", "//a/@href"), told apart by how it begins: an XPath begins with /, ./, ( or @, or is written after "xpath:". respect_tdm: answer tdm_reserved when the site reserves its text and data mining rights (TDMRep).
Returns {"ok", "url", "values", "count"}, and "fetch" for a URL: values are {"value", "where"}, the text or attribute read, spaces collapsed, links resolved, and the XPath of its element; count is how many the selector gave. A selector that cannot be read is bad_input naming it; one that gives nothing is ok with no values. Past 75,000 bytes the last values are left out, counted in values_left_out. Use it to try the selectors compile_extractor's select takes.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| selector | Yes | CSS, with ::text for an element's own text and ::attr(name) for an attribute ("span.price::text", "a::attr(href)"), or XPath ("//h1", "//a/@href"), told apart by how it begins: an XPath begins with /, ./, ( or @, or is written after "xpath:". | |
| html_or_url | Yes | an http(s) URL to fetch, or the HTML itself. | |
| respect_tdm | No | answer tdm_reserved when the site reserves its text and data mining rights (TDMRep). |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| ok | Yes | ||
| url | No | ||
| count | No | ||
| error | No | ||
| fetch | No | ||
| values | No | ||
| values_left_out | No |