word_tokenize
Split text into word tokens, handling contractions, hyphenated words, numbers, and punctuation.
Instructions
Split text into word tokens. Handles contractions, hyphenated words, numbers, and punctuation.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"text": "<text>"
},
"name": "word_tokenize"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |