Find a phrase in the scanned text of every archive
search_insideSearch inside texts across Internet Archive, Library of Congress, and other archives simultaneously to locate where a phrase appears, with page numbers and source-labelled excerpts.
Instructions
Search the machine-read text of every archive this server reads that holds any, at the same time, and get one list of where a phrase was printed. The archives behind this tool hold different material, so the list is additive: it is the places a phrase appears in each corpus, put together, rather than the same question answered twice. 'per_source' names what each corpus is. A match carries 'page_number', which is a real leaf on an archive whose index holds one and null on an archive whose index holds none. That null is the index having no leaf, never a page this server dropped, and no page is ever invented. A match also carries 'excerpt_kind'. 'passage' is the text around the words that matched. 'page_opening' is the start of the page, sent because the machine-read text that came back stops before the searched words appear, so it does not carry the match. The notes say how many excerpts are openings. Every count is that archive's own and counts something of its own: documents in one place, leaves in another. They are never added together, and there is no total across archives. Rows are interleaved one archive at a time. Nothing ranks them against each other and nothing orders them by date, because a year is measured on different things in each archive. Matches whose excerpt carries the searched words are placed before matches whose excerpt is a 'page_opening' and carries them nowhere. That rests on what each row states about its own excerpt rather than on any score, no match is ever dropped for it, and the interleaving holds inside each of the two groups. The archives read the words given in different ways, and 'per_source' says which each one does. An index that answers only where every word appears returns nothing for a question written as a sentence, even on a work it holds several copies of; an index that scores the words instead answers such a question with the pages it ranks highest, which can carry only some of them. Either way it is the words: a character that is neither a letter nor a digit is no word to an index, and 'non_word_characters' lists any the query carried. Shorter and differently spelled wordings are therefore derived from the query and asked for their union, which costs nothing extra when the words as asked already answer. Every wording sent is named in 'per_source' with what it returned, every match carries the wording that returned it in 'found_by_query', and 'fan_out' turns the derivation off. Use search_items for a work by its title, its creator or its subject: this tool reads the text on the pages and knows nothing of a catalogue, so a title given here finds every book that happens to mention it and misses the book itself. Answers take several seconds, because one of the archives publishes a request ceiling this server keeps to. A slow answer is the pacing, not a stall.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Which page of matches, from 1. Each archive is paged separately. | |
| limit | No | Matches to take from each archive, so one archive cannot fill the whole list. The text block shows about six matches; the rest of what comes back is in the structured output. | |
| query | Yes | Words, or a phrase in double quotes such as '"call me ishmael"'. | |
| fan_out | No | Whether to derive shorter and differently spelled wordings from the query and ask each archive for the union of what they return. A question written as a sentence returns nothing where every word given has to appear, and the rows an index scores highest where it does not. An archive is asked a derived wording only when the words as asked did not return as many rows as 'limit', so a query that works costs one request. Set false to send exactly the words given. 'per_source' names every wording, sent or not, and each match names the one that returned it. | |
| sources | No | Archives to ask, by id. Left out, every archive that reads machine-read text is asked, which is the point of this tool. | |
| max_excerpt_chars | No | Budget for one passage. Read it together with 'max_excerpts_per_match': the size of the answer is the product of the two, the limit, and the number of archives. | |
| max_excerpts_per_match | No | Passages to keep per match. A long work matches in several places, and the later ones rarely say anything the first did not. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| hits | Yes | ||
| page | Yes | ||
| notes | Yes | ||
| order | Yes | How the list was built, in words. | |
| query | Yes | ||
| hit_count | Yes | Matches in this answer, across every archive. It is a count of what came back, never a total of what exists. | |
| per_source | Yes | ||
| queries_run | Yes | Requests this server sent for this answer, counting every wording on every archive. Each archive's own wordings are in 'per_source'. | |
| excerpt_kinds | Yes | How many excerpts here are of each kind. | |
| non_word_characters | Yes | Characters in the query that are neither letters nor digits. These indexes answer on words, so a match here can carry none of them, and 'requires_every_word' covers the words that were given rather than these. |