Normalise Arabic text by removing diacritics, tatweel, and (optionally) unifying letter variants.

• Unicode-normalises to NFC.

• Optionally removes Arabic diacritics (harakat / tashkil).

• Optionally removes the tatweel (kashida) elongation character.

• Optionally collapses alef/yeh/teh-marbuta variants (off by default; lossy).

Args: text: The Arabic (or mixed) text to normalise. strip_diacritics: Remove harakat / tashkil marks. Defaults to True. strip_tatweel: Remove the tatweel (kashida) character. Defaults to True. normalise_letters: Collapse alef/yeh/teh-marbuta variants. Default False.

Returns: The normalised text.

Remove Arabic diacritics (tashkeel) and, optionally, the tatweel.

Args: text: The Arabic (or mixed) text to clean. strip_tatweel: Also remove the tatweel character. Defaults to True.

Returns: The text with diacritics (and optionally tatweel) removed.

Transliterate Arabic text into Latin characters.

Uses a documented, deterministic Arabic -> Latin scheme (loosely DIN 31635 / ALA-LC, simplified to ASCII digraphs). See :func:arabic_tools.transliterate for the full documented limitations.

Args: text: The Arabic text to transliterate.

Returns: A dict with the transliterated string and the scheme name.

Guess the Arabic dialect using a transparent keyword heuristic.

This is a rule-based heuristic, NOT a trained classifier. It counts hand-picked marker words per dialect (Egyptian, Levantine, Gulf, Maghrebi, MSA) and returns the best match with a crude confidence. See :func:arabic_tools.detect_dialect for the full documented limitations.

Args: text: The Arabic text to classify.

Returns: A dict with the predicted dialect, label, crude confidence, per-dialect scores, and a note documenting that this is a heuristic.

Count basic statistics: whitespace tokens, characters, Arabic characters.

"Tokens" means whitespace-delimited words (not an LLM subword tokenizer).

Args: text: The text to measure.

Returns: A dict with token, character, no-space character, and Arabic-character counts.