word_tokenize
Split text into word tokens to process natural language. Handles contractions, hyphenated words, numbers, and punctuation for text analysis.
Instructions
Split text into word tokens. Handles contractions, hyphenated words, numbers, and punctuation.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |