UZ-ByT5
Explainable Uzbek morpheme analyzer
A byte-level ByT5 fine-tune that splits Uzbek words into four grammatical morpheme classes and restores the dictionary form of altered stems — a reconstruction task subword tokenizers cannot perform.
- 98.9%exact match
- 99.4%morpheme F1
- 246Ktraining examples
- Generated the corpus instead of scraping it: Uzbek morphophonological rules applied forward over a 93K-root lexicon and a 354-affix taxonomy, producing 246K examples with guaranteed-correct labels.
- Diagnosed an fp16 divergence specific to the T5 family — training collapsed to 0% accuracy — and resolved it in fp32.
- Shipped with a deterministic rule-based fallback, a human-correction layer, Uzbek TTS, 31 unit tests in CI and a portable Windows build.
PyTorch · ByT5 · Transformers · Streamlit
