Split text into word tokens. Handles contractions, hyphenated words, numbers, and punctuation.

Split text into sentences. Handles abbreviations (Mr., Dr., etc.) and tricky boundaries.

Generate n-grams from a list of tokens. Returns list of n-gram lists.

Generate character-level n-grams from text.

Flesch Reading Ease score. 90-100=very easy, 60-69=standard, 0-29=very confusing.

Flesch-Kincaid Grade Level. Returns US school grade level needed to understand text.

Gunning Fog Index. Estimates years of formal education needed to understand text.

Coleman-Liau Index. Grade level based on characters per word and sentences per word.

Automated Readability Index (ARI). Grade level from character and word counts.

SMOG Grade. Best for healthcare/medical texts. Counts polysyllabic words.

Estimate syllable count for a single word using heuristics.

Comprehensive reading level: grade level, label (elementary/middle/high school/college/graduate), and all readability scores.

Compound sentiment score from -1 (negative) to 1 (positive). VADER-style with built-in 2000+ word lexicon.

Classify text as 'positive', 'negative', or 'neutral'.

Per-sentence sentiment breakdown. Returns list of {sentence, score, label}.

Sentiment around specific topics/aspects. Finds sentences mentioning each aspect and averages their sentiment.

Extract keywords using TF-IDF computed from scratch. Pass multiple docs for best results.

RAKE keyword extraction (Rapid Automatic Keyword Extraction). Finds multi-word key phrases.

Most frequent words excluding stopwords. Returns [{word, count}].

Most frequent n-grams (phrases). Default bigrams. Returns [{phrase, count}].

Jaccard similarity (word-level set overlap). 0=no overlap, 1=identical word sets.

Cosine similarity using bag-of-words vectors. 0=orthogonal, 1=identical.

Levenshtein edit distance. Minimum single-character edits to transform s1 into s2.

Normalized edit distance on 0-1 scale. 0=identical, 1=completely different.

Length of longest common subsequence (LCS) between two strings.

Remove English stopwords (500+ built-in) from text.

Remove all punctuation from text.

Remove all numbers from text.

Remove URLs from text.

Remove email addresses from text.

Remove HTML tags from text.

Collapse multiple whitespace into single spaces.

Convert text to lowercase.

Porter stemmer from scratch. Reduce word to its stem (e.g., 'running' -> 'run').

Configurable cleaning pipeline. Steps: html, urls, emails, numbers, punctuation, stopwords, whitespace, lowercase.

Detect language from text. Returns top 5 matches with confidence scores. Supports 18 languages.

Detect character encoding type: ASCII, Latin, Cyrillic, CJK, Arabic, etc.

Confidence that text is English (0-1 scale).

Count words in text.

Count sentences in text.

Count paragraphs in text (separated by blank lines).

Average word length in characters.

Average sentence length in words.

Extract the best N sentences as a summary. Scores by position, keyword frequency, length, and title overlap.

Comprehensive text stats: words, sentences, paragraphs, reading time, readability scores, language.