Inspect Terms
document.inspect_termsInspect tokenization patterns by viewing emitted trigrams or sparse n-grams with offsets and rarity weights for a file or text snippet.
Instructions
Show the trigrams or sparse n-grams that would be emitted for a file or text snippet, including offsets and rarity weights.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | Choose the tokenization mode to inspect. | |
| path | No | Absolute path to the source file to inspect. | |
| text | No | Inline text snippet to inspect instead of reading a file. | |
| maxTerms | No | Maximum number of emitted terms to return. Defaults to 256. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | ||
| notes | Yes | ||
| terms | Yes | ||
| termCount | Yes |