API reference

Top-level API

indic_normalizer — a lightweight TTS pre-normalizer for the 22 scheduled Indian languages (+ English).

Public API:

from indic_normalizer import Normalizer, normalize
normalize("भारत 1947 में स्वतंत्र हुआ", lang="hi")

See NormalizerConfig for options.

class indic_normalizer.Normalizer(lang: str = 'en', config: NormalizerConfig | None = None, **kwargs)[source]

Bases: object

Lightweight TTS pre-normalizer for Indic languages + English.

normalize(text: str) → str[source]

Normalize text to spoken form.

Stages: control/unicode cleanup -> LaTeX spans -> escape stripping -> tag protection -> parenthetical removal -> semiotic-class handlers -> tag restore -> whitespace/punctuation tidy (NFC output).

indic_normalizer.normalize(text: str, lang: str = 'en', **kwargs) → str[source]

One-shot convenience wrapper.

class indic_normalizer.NormalizerConfig(lang: str = 'en', number_lang: str | None = None, force: bool = False, default_number_lang: str = 'en', strip_parentheses: bool = True, keep_square_brackets: bool = True, keep_angle_brackets: bool = True, strip_escapes: bool = True, latex: bool = True, latex_verbosity: str = 'natural', detect_years: bool = True, year_range: Tuple[int, int] = (1100, 2099), detect_positions: bool = True, detect_roman: bool = False, emit_variations: bool = False)[source]

Bases: object

Options controlling normalization.

lang

The sentence’s (regional) language. Drives lexicon selection and the verbalization language for native-script digits.

Type:

str

number_lang

If set, force ALL numbers to be verbalized in this language, overriding script-based resolution. force=True is a shorthand that sets this to lang.

Type:

str | None

force

Shorthand: force all numbers into lang.

Type:

bool

default_number_lang

Language used for ASCII/Arabic (0-9) digits when not forced. Defaults to English, matching common Indian TTS usage.

Type:

str

strip_parentheses

Remove (...) spans (content included).

Type:

bool

keep_square_brackets / keep_angle_brackets

Preserve [...] / <...> spans verbatim (SSML / prosody markers).

strip_escapes

Decode & strip escape sequences and control chars.

Type:

bool

latex

Convert LaTeX math spans to spoken English (auto-detect).

Type:

bool

latex_verbosity

“natural” or “explicit”.

Type:

str

detect_years

Enable year-style reading heuristic.

Type:

bool

year_range

Inclusive [lo, hi] range treated as candidate years.

Type:

Tuple[int, int]

detect_positions

Read 3-4 digit numbers after position words (room/page/flight/gate/bus) in pairing style: “room 225” -> “room two twenty five” (English only).

Type:

bool

emit_variations

If True, normalize may annotate alt readings (currently used by the number layer for debugging/inspection).

Type:

bool

resolve_number_lang(native: bool) → str[source]

Pick the language to verbalize a number in.

  1. forced number_lang wins;

  2. else native-script digits -> the sentence language;

  3. else (ASCII digits) -> default_number_lang (English).

indic_normalizer.latex_to_speech(expr, verbosity='natural')[source]

Convert raw LaTeX (without delimiters) to spoken English.

Never raises: on failure, degrades to a stripped reading.

indic_normalizer.normalize(text: str, lang: str = 'en', **kwargs) → str[source]

One-shot convenience wrapper.

class indic_normalizer.Normalizer(lang: str = 'en', config: NormalizerConfig | None = None, **kwargs)[source]

Bases: object

Lightweight TTS pre-normalizer for Indic languages + English.

normalize(text: str) → str[source]

Normalize text to spoken form.

Stages: control/unicode cleanup -> LaTeX spans -> escape stripping -> tag protection -> parenthetical removal -> semiotic-class handlers -> tag restore -> whitespace/punctuation tidy (NFC output).

Pipeline

The main normalization pipeline.

class indic_normalizer.normalizer.Normalizer(lang: str = 'en', config: NormalizerConfig | None = None, **kwargs)[source]

Bases: object

Lightweight TTS pre-normalizer for Indic languages + English.

normalize(text: str) → str[source]

Normalize text to spoken form.

Stages: control/unicode cleanup -> LaTeX spans -> escape stripping -> tag protection -> parenthetical removal -> semiotic-class handlers -> tag restore -> whitespace/punctuation tidy (NFC output).

indic_normalizer.normalizer.normalize(text: str, lang: str = 'en', **kwargs) → str[source]

One-shot convenience wrapper.

Configuration for the normalizer pipeline.

class indic_normalizer.config.NormalizerConfig(lang: str = 'en', number_lang: str | None = None, force: bool = False, default_number_lang: str = 'en', strip_parentheses: bool = True, keep_square_brackets: bool = True, keep_angle_brackets: bool = True, strip_escapes: bool = True, latex: bool = True, latex_verbosity: str = 'natural', detect_years: bool = True, year_range: Tuple[int, int] = (1100, 2099), detect_positions: bool = True, detect_roman: bool = False, emit_variations: bool = False)[source]

Bases: object

Options controlling normalization.

lang

The sentence’s (regional) language. Drives lexicon selection and the verbalization language for native-script digits.

Type:

str

number_lang

If set, force ALL numbers to be verbalized in this language, overriding script-based resolution. force=True is a shorthand that sets this to lang.

Type:

str | None

force

Shorthand: force all numbers into lang.

Type:

bool

default_number_lang

Language used for ASCII/Arabic (0-9) digits when not forced. Defaults to English, matching common Indian TTS usage.

Type:

str

strip_parentheses

Remove (...) spans (content included).

Type:

bool

keep_square_brackets / keep_angle_brackets

Preserve [...] / <...> spans verbatim (SSML / prosody markers).

strip_escapes

Decode & strip escape sequences and control chars.

Type:

bool

latex

Convert LaTeX math spans to spoken English (auto-detect).

Type:

bool

latex_verbosity

“natural” or “explicit”.

Type:

str

detect_years

Enable year-style reading heuristic.

Type:

bool

year_range

Inclusive [lo, hi] range treated as candidate years.

Type:

Tuple[int, int]

detect_positions

Read 3-4 digit numbers after position words (room/page/flight/gate/bus) in pairing style: “room 225” -> “room two twenty five” (English only).

Type:

bool

emit_variations

If True, normalize may annotate alt readings (currently used by the number layer for debugging/inspection).

Type:

bool

resolve_number_lang(native: bool) → str[source]

Pick the language to verbalize a number in.

  1. forced number_lang wins;

  2. else native-script digits -> the sentence language;

  3. else (ASCII digits) -> default_number_lang (English).

indic_normalizer.config.SUPPORTED_LANGS

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable’s items.

If the argument is a tuple, the return value is the same object.

Semiotic-class handlers

Ordered registry of semiotic-class handlers.

class indic_normalizer.classes.Context(cfg: NormalizerConfig)[source]

Bases: object

Per-run state shared by all handlers: the config plus number-language and lexicon resolution.

property sentence_lex: Lexicon

Lexicon of the sentence language (cfg.lang).

resolve(sample: str) → Tuple[str, Lexicon][source]

Resolve the verbalization language + lexicon for a numeric token.

sample is the raw matched digit text; native-script digits route to the sentence language, ASCII digits to English (unless forced).

indic_normalizer.classes.apply_all(text: str, ctx: Context, enable_roman: bool = False) → str[source]

Run every enabled handler over text in registry (priority) order.

Shared context and helpers for semiotic-class handlers.

Each handler is a callable (text, ctx) -> text that rewrites the spans it recognises into spoken words. Handlers run in a fixed priority order (specific -> general); because each handler replaces its matches with words, later numeric handlers never re-match already-verbalized spans.

class indic_normalizer.classes.base.Context(cfg: NormalizerConfig)[source]

Bases: object

Per-run state shared by all handlers: the config plus number-language and lexicon resolution.

property sentence_lex: Lexicon

Lexicon of the sentence language (cfg.lang).

resolve(sample: str) → Tuple[str, Lexicon][source]

Resolve the verbalization language + lexicon for a numeric token.

sample is the raw matched digit text; native-script digits route to the sentence language, ASCII digits to English (unless forced).

Semiotic-class handlers (regex tagger -> verbalizer), in priority order.

indic_normalizer.classes.handlers.web_(text: str, ctx: Context) → str[source]

Speak emails and URLs token-by-token (“at”, “dot”, “slash”, digits digit-wise); the https:// prefix is dropped.

indic_normalizer.classes.handlers.abbrev_(text: str, ctx: Context) → str[source]

Expand titles and abbreviations (“Dr.”, “Pvt. Ltd.”, “etc.”, “vs.”, “No. 5”, “St.” Saint/Street heuristic) and space out dotted acronyms (“U.S.A.” -> “U S A”).

indic_normalizer.classes.handlers.alphanumeric_split(text: str, ctx: Context) → str[source]

Split Latin-letter/digit boundaries (“COVID19” -> “COVID 19”) so the embedded number verbalizes while the Latin part stays verbatim.

indic_normalizer.classes.handlers.time_(text: str, ctx: Context) → str[source]

HH:MM(:SS) with optional am/pm: “10:30 am” -> “ten thirty am”.

indic_normalizer.classes.handlers.ratio_(text: str, ctx: Context) → str[source]

Colon pairs the time handler rejected: “3:2” -> “three to two”.

indic_normalizer.classes.handlers.date_(text: str, ctx: Context) → str[source]

Numeric dates (d/m/yyyy, d-m-yyyy, yyyy-mm-dd, d.m.yyyy). An invalid day/month pair first tries US month/day order; a hopeless triple reads as plain numbers instead of leaking to the fraction handler.

indic_normalizer.classes.handlers.textdate_(text: str, ctx: Context) → str[source]

English textual dates (“15 August 1947”, “Aug 15, 1947”, “5 June”): ordinal day, expanded month name, year-style year.

indic_normalizer.classes.handlers.money_(text: str, ctx: Context) → str[source]

Currency amounts: prefix symbol/code (”₹1,234.50”, “Rs 150”), scale words (”₹5 lakh”), signs, singular units (”₹1” -> “one rupee”), and suffix forms (“100₹”, “250 rs”).

indic_normalizer.classes.handlers.percent_(text: str, ctx: Context) → str[source]

“12.5%” -> “twelve point five percent” (optional leading sign).

indic_normalizer.classes.handlers.ids_(text: str, ctx: Context) → str[source]

Indian ID formats spelled letter-by-letter with digit-wise digits: PAN (ABCDE1234F), IFSC (SBIN0001234), vehicle plates (KA 01 AB 1234), and “PIN <6 digits>”.

indic_normalizer.classes.handlers.bp_(text: str, ctx: Context) → str[source]

Blood pressure (“120/80 mmHg”, or “BP 120/80”) -> “one hundred and twenty over eighty (millimeters of mercury)”.

indic_normalizer.classes.handlers.measure_(text: str, ctx: Context) → str[source]

Number + unit (“5kg”, “37.5°C”, “2.4 GHz”) -> spoken unit words. Unit matching is case-sensitive so “5G” is never five grams.

indic_normalizer.classes.handlers.native_scale_(text: str, ctx: Context) → str[source]

ASCII number + native-script scale word (“2 करोड़”) reads the number in the scale word’s language: “दो करोड़”.

indic_normalizer.classes.handlers.range_(text: str, ctx: Context) → str[source]

“N-M” ranges up to 4 digits per side (“1939-1945”, “10-15%”, “5-10 kg”) -> “N to M” (year-style sides where plausible); longer sides fall through to the phone handler.

indic_normalizer.classes.handlers.phone_(text: str, ctx: Context) → str[source]

Phone-shaped digit runs -> digit-by-digit (“+91 98765 43210”); shape-gated (see _looks_like_phone()) so year lists and large cardinals pass through to the number handler.

indic_normalizer.classes.handlers.dotted_(text: str, ctx: Context) → str[source]

Digit runs with >= 2 dots: IPv4 reads digit-wise with “dot”; versions (“3.11.4”) read as cardinals joined by the decimal-point word.

indic_normalizer.classes.handlers.cricket_(text: str, ctx: Context) → str[source]

“287/5” near a cricket word -> “two hundred and eighty seven for five”; without the context word the fraction handler keeps it.

indic_normalizer.classes.handlers.fraction_(text: str, ctx: Context) → str[source]

“a/b” -> a named fraction (“3/4” -> “three quarters”) or “a over b” (English); other languages join with the “and” connector.

indic_normalizer.classes.handlers.decimal_(text: str, ctx: Context) → str[source]

“3.14” -> “three point one four” (fraction digits read one by one).

indic_normalizer.classes.handlers.scientific_(text: str, ctx: Context) → str[source]

“1.5e10” -> “one point five times ten to the power ten” (English-only wording; other resolved languages pass through).

indic_normalizer.classes.handlers.code_(text: str, ctx: Context) → str[source]

Alphanumeric codes with >= 2 letter/digit transitions (“AB123CD”, “6E204”) spell out: letters one by one, digits digit-wise. Tokens with a single transition (“32A”, “B12”) keep the natural reading.

indic_normalizer.classes.handlers.decade_(text: str, ctx: Context) → str[source]

“1990s” / “1980’s” / “90s” -> “nineteen nineties” / “nineties” (English idiom; non-decades like “1994s” are left alone).

indic_normalizer.classes.handlers.ordinal_(text: str, ctx: Context) → str[source]

English “21st” -> “twenty first”; native suffix ordinals (“5वाँ”) speak the sentence language regardless of digit script.

indic_normalizer.classes.handlers.position_(text: str, ctx: Context) → str[source]

3-4 digit numbers after a position word (room/page/flight/gate/bus) read in pairing style: “room 225” -> “room two twenty five”, “flight 6204” -> “flight sixty two oh four”. Gated by detect_positions.

indic_normalizer.classes.handlers.number_(text: str, ctx: Context) → str[source]

Bare integers: year-style for plausible bare 4-digit years, digit-wise for leading zeros (“007”), else cardinal; leading +/- spoken.

indic_normalizer.classes.handlers.symbol_(text: str, ctx: Context) → str[source]

Standalone symbols between spaces (& @ + = < >) -> words.

indic_normalizer.classes.handlers.roman_ctx_(text: str, ctx: Context) → str[source]

Roman numerals after a title word (“Chapter IV”, “Class X”, “World War II”) -> cardinals. Always on: the trigger word makes even single-letter numerals safe.

indic_normalizer.classes.handlers.roman_(text: str, ctx: Context) → str[source]

Bare roman numerals anywhere -> cardinals. Opt-in via detect_roman — risky on ordinary uppercase words.

Number verbalization

Number verbalization built on the vendored engine.

The vendored _numengine handles cardinals for all 22 scheduled languages + English. This module wraps it with graceful fallbacks and adds the readings the engine does not provide: decimals, digit-strings, ordinals, and year-style.

indic_normalizer.numbers.core.cardinal(value, lang: str = 'en') → str[source]

Return the primary cardinal reading of a non-negative integer.

Never raises; on engine failure falls back to a digit-by-digit reading.

indic_normalizer.numbers.core.cardinal_variations(value, lang: str = 'en') → List[str][source]

All cardinal readings the engine offers for value (at least one).

indic_normalizer.numbers.core.split_digits(digit_str: str, lang: str = 'en') → str[source]

Read each digit separately: ‘420’ -> ‘four two zero’.

indic_normalizer.numbers.core.decimal(number_str: str, lang: str, lex) → str[source]

Read a decimal: integer part as cardinal, fractional part digit-by-digit.

indic_normalizer.numbers.core.ordinal(value: int, lang: str, lex) → str[source]

Ordinal reading: English forms for en, else the lexicon’s rule (irregular map, then cardinal + suffix).

indic_normalizer.numbers.core.year(value: int, lang: str, lex) → str[source]

Year-style reading.

English-style pairing (nineteen forty-seven) is used when the lexicon opts in (year_pairing); otherwise the full cardinal is used, which is the natural reading for most Indic languages.

indic_normalizer.numbers.core.english_ordinal(value: int) → str[source]

English ordinal words: 21 -> “twenty first”, 12 -> “twelfth”.

Unicode numeral handling for Indic scripts.

We rely on unicodedata for decoding any Unicode decimal digit to its value, and keep an explicit script -> (digit range, language hint) table so we can (a) recognise native-script digits, and (b) name the script of a run of digits. Language resolution for verbalization only needs: “is this digit ASCII, or native-script?” (see indic_normalizer.normalizer).

indic_normalizer.numerals.is_ascii_digit(ch: str) → bool[source]

True if ch is one of “0”-“9”.

indic_normalizer.numerals.is_native_digit(ch: str) → bool[source]

True if ch is a non-ASCII Unicode decimal digit (Indic/Perso-Arabic).

indic_normalizer.numerals.digit_value(ch: str) → int | None[source]

Numeric value of a single digit char (ASCII or native), else None.

indic_normalizer.numerals.to_ascii_digits(s: str) → str[source]

Convert every native digit in s to its ASCII equivalent.

Non-digit characters are passed through unchanged.

indic_normalizer.numerals.digits_are_native(s: str) → bool[source]

True if the run of digit characters in s contains any native digit.

indic_normalizer.numerals.script_of_digits(s: str) → str | None[source]

Name of the digit script used in s (first native digit wins).

Lexicon

Lexicon registry: build and cache one Lexicon per language.

class indic_normalizer.lexicon.Lexicon(lang: str, decimal_point: str = 'point', negative: str = 'minus', connector_and: str = 'and', percent: str = 'percent', range_to: str = 'to', year_pairing: bool = False, year_hundred_word: str = 'hundred', year_oh_word: str = 'oh', ordinal_suffix: str = '', ordinal_irregular: Dict[int, str]=<factory>, symbols: Dict[str, str]=<factory>, currency: Dict[str, ~typing.Tuple[str, str]]=<factory>, units: Dict[str, str]=<factory>, months: Dict[int, str]=<factory>, year_trigger_words: Tuple[str, ...]=(), date_trigger_words: Tuple[str, ...]=(), review: Tuple[str, ...]=())[source]

Bases: object

Per-language glue words the number engine cannot provide: decimal point, percent, currency units, months, ordinal rules, connectors. Fields left at their defaults fall back to English wording; review names entries still awaiting native-speaker verification.

ordinal(value: int, cardinal_fn) → str[source]

Default ordinal formation: irregular map, else cardinal + suffix.

indic_normalizer.lexicon.get_lexicon(lang: str) → Lexicon[source]

Return the cached Lexicon for lang (built on first use).

indic_normalizer.lexicon.list_review_flags() → List[str][source]

Return "<lang>: <field>" for every low-confidence lexicon entry.

Per-language lexicon: the glue words the number engine cannot provide.

Each language ships a Lexicon instance in lexicon/data/<lang>.py. Entries that could not be verified by a native speaker are marked in the review set (their keys) so they can be surfaced for follow-up.

class indic_normalizer.lexicon.base.Lexicon(lang: str, decimal_point: str = 'point', negative: str = 'minus', connector_and: str = 'and', percent: str = 'percent', range_to: str = 'to', year_pairing: bool = False, year_hundred_word: str = 'hundred', year_oh_word: str = 'oh', ordinal_suffix: str = '', ordinal_irregular: Dict[int, str]=<factory>, symbols: Dict[str, str]=<factory>, currency: Dict[str, ~typing.Tuple[str, str]]=<factory>, units: Dict[str, str]=<factory>, months: Dict[int, str]=<factory>, year_trigger_words: Tuple[str, ...]=(), date_trigger_words: Tuple[str, ...]=(), review: Tuple[str, ...]=())[source]

Bases: object

Per-language glue words the number engine cannot provide: decimal point, percent, currency units, months, ordinal rules, connectors. Fields left at their defaults fall back to English wording; review names entries still awaiting native-speaker verification.

ordinal(value: int, cardinal_fn) → str[source]

Default ordinal formation: irregular map, else cardinal + suffix.

Preprocessing

Artifact stripping: control chars, escape sequences, whitespace, unicode.

Split into two phases so LaTeX (which is full of backslashes) can be extracted in between:

  • clean_controls() — safe to run first; NFC-normalizes, drops control & zero-width characters, and collapses whitespace. Leaves backslashes intact.

  • strip_escapes() — run AFTER LaTeX extraction; decodes literal \uXXXX / \xHH sequences and removes leftover escape artifacts (\n, \t, stray backslashes).

indic_normalizer.preprocess.artifacts.clean_controls(text: str, collapse_newlines: bool = True) → str[source]

NFC-normalize, drop control/zero-width chars, collapse whitespace.

indic_normalizer.preprocess.artifacts.strip_escapes(text: str) → str[source]

Decode/strip escape-sequence artifacts. Run after LaTeX extraction.

Bracket handling.

  • (...) — removed entirely (content included), honouring nesting.

  • [...] and <...> — preserved verbatim; they are SSML / prosody markers or tags. They are masked with private-use sentinels before the rest of the pipeline runs, then restored unchanged at the very end.

indic_normalizer.preprocess.brackets.protect_tags(text: str, square: bool = True, angle: bool = True) → Tuple[str, List[str]][source]

Replace [...] / <...> spans with sentinels; return (masked, store).

indic_normalizer.preprocess.brackets.restore_tags(text: str, store: List[str]) → str[source]

Re-insert the spans masked by protect_tags() (its inverse).

indic_normalizer.preprocess.brackets.strip_parentheses(text: str) → str[source]

Remove balanced (...) spans including nested content.

Only balanced pairs are removed (innermost-first); an unmatched “(” or “)” is kept as a literal so it cannot swallow the rest of the utterance.

Lightweight Unicode script detection (used for diagnostics / future hooks).

indic_normalizer.preprocess.scripts.char_script(ch: str) → str | None[source]

Name of the script block ch belongs to, else None.

indic_normalizer.preprocess.scripts.dominant_script(text: str) → str | None[source]

Most frequent non-Latin script in text; “latin” if only Latin is present, None if no known script at all.

LaTeX-to-speech

Pure-Python, English-only LaTeX-to-speech engine.

Public API

latex_to_speech(expr, verbosity=”natural”) -> str convert_spans(text, verbosity=”natural”) -> str LATEX_SPAN_PATTERN (compiled re.Pattern)

indic_normalizer.latex.latex_to_speech(expr, verbosity='natural')[source]

Convert raw LaTeX (without delimiters) to spoken English.

Never raises: on failure, degrades to a stripped reading.

indic_normalizer.latex.convert_spans(text, verbosity='natural')[source]

Replace every LaTeX span in text with its spoken reading.

Each span becomes " <spoken> " (space-padded). Non-LaTeX text is untouched. Never raises.

Chemistry: mhchem ce{…} rendering plus a bare-formula heuristic.

Element symbols are read letter-by-letter (uppercase letters spoken as the letter), digit counts as English cardinals, per the approved examples:

\ce{H2O}            -> "H two O"
\ce{2H2 + O2 -> 2H2O} -> "two H two plus O two yields two H two O"
\ce{SO4^2-}         -> "S O four two minus"
indic_normalizer.latex.chemistry.read_ce(inner)[source]

Read the contents of a ce{…} span.

indic_normalizer.latex.chemistry.looks_like_formula(token)[source]

Return True if token looks like a chemical formula.

Guards against firing on ordinary words / all-caps acronyms (NASA, IPL): requires either a digit or at least two element groups with a lowercase letter present.

indic_normalizer.latex.chemistry.read_formula(token)[source]

Read a bare chemical formula token (e.g. H2O).