Text Clean
text_cleanClean and normalise messy text before embedding, indexing or prompting: repairs mojibake (UTF-8 read as Windows-1252), strips control and invisible characters (zero-width, bidi overrides used in Trojan Source attacks, soft hyphens, BOM), normalises exotic whitespace, curly quotes, dashes, ligatures and full-width forms, re-joins words hyphenated across line breaks, optionally unwraps hard-wrapped lines, strips HTML, decodes entities, removes emoji / URLs / emails, transliterates to ASCII or lowercases. Reports how many fixes of each kind were made. Offline; 500 KB. Price: $0.001 USDC per successful call (x402, Solana). Same tool over HTTPS: POST https://scorptech.it.com/api/x402/v1/text-clean
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"text": "<text>"
},
"name": "text_clean"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text to clean | |
| ascii | No | Transliterate Latin accents to ASCII (é -> e) | |
| fullwidth | No | Full-width ASCII forms to ASCII | |
| ligatures | No | Expand fi fl ff ffi ffl ſt st | |
| lowercase | No | Lowercase everything | |
| stripHtml | No | Remove HTML tags (script/style content dropped) | |
| removeUrls | No | Remove http(s) URLs | |
| dehyphenate | No | Join words hyphenated across line breaks (infor-\nmation) | |
| fixMojibake | No | Repair UTF-8 decoded as Windows-1252 (Café -> Café, ’ -> ’) | |
| removeEmoji | No | Remove emoji and pictographs | |
| unwrapLines | No | Join hard-wrapped lines inside paragraphs | |
| removeEmails | No | Remove email addresses | |
| stripControl | No | Remove C0/C1 control characters (keeps tab / newline) | |
| decodeEntities | No | Decode HTML entities (& -> &) | |
| stripInvisible | No | Remove zero-width, bidi override (Trojan Source), soft hyphen and BOM characters | |
| normalizeDashes | No | En / em / figure dashes and minus sign to "-" | |
| normalizeQuotes | No | Curly quotes / primes to ASCII quotes and … to ... | |
| normalizeWhitespace | No | Tabs and exotic spaces to plain spaces, collapse runs, trim lines, max one blank line |