MameLoshnLM: the first open Yiddish LLM (and what it means for Yiddish tech)
The first real Yiddish language model — and why it matters
Two releases landed on Hugging Face that anyone working on Yiddish tech should know about: MameLoshnLM, an open 8B-parameter language model built specifically for Yiddish, and the paper behind it — MameLoshnLM: Yiddish Language Model and Evaluation Benchmark (arXiv 2608.05850). Code is at github.com/katzurik/MameLoshnLM.
To the authors' knowledge it is the first large generative language model adapted specifically to Yiddish.
What they built
Three artifacts:
- Oytser — a high-quality Yiddish pretraining corpus. This matters because the paper's core finding is that existing multilingual corpora are poor proxies for Yiddish: they contain substantial amounts of noisy, machine-translated, and misclassified text. Oytser combines contemporary web-native Yiddish with an extensive collection of literary materials.
- Kashes — a new multi-task Yiddish benchmark spanning translation (including Kashes-mt, a new translation task they created), linguistic analysis, information extraction, and language understanding.
- MameLoshnLM — continued pretraining of Llama 3.1 8B on Oytser (one epoch, small proportion of English interleaved to mitigate catastrophic forgetting, unchanged tokenizer, bf16, 1,024-token training context, 8-bit AdamW).
The results: native data beats scale
Across the benchmark, MameLoshnLM outperforms same-scale open baselines (Llama 3.1 8B, Qwen3 8B) on Yiddish — average score 62.6 vs 56.8 and 54.7. Highlights (5-shot):
| Task | MameLoshnLM | Llama 3.1 8B | Qwen3 8B |
|---|---|---|---|
| POS tagging (acc.) | 88.6 | 86.9 | 85.9 |
| Lemmatization (change acc.) | 31.9 | 19.7 | 21.6 |
| NER — EHRI (F1) | 41.3 | 34.2 | 20.8 |
| MT Eng→Yid, FLORES+ (COMET) | 78.5 | 64.8 | 46.5 |
| MT Eng→Yid, Kashes-mt (COMET) | 75.3 | 59.6 | 45.3 |
| MT Yid→Eng, FLORES+ (COMET) | 87.2 | 82.2 | 79.1 |
The gains are not just quantitative. Relative to general-purpose multilingual models, MameLoshnLM better captures the lexical and morphological patterns that define the language — idioms, style, cultural voice. The authors name the failure mode directly: noisy web-scale multilingual data flattens low-resource languages.
What it means for Yiddish ASR/TTS work
This is highly relevant to anyone building the surrounding stack (transcription, speech, learning tools):
- A real generative foundation for Yiddish finally exists — usable for completion-style prompting, few-shot in-context learning, and as a base for task-specific fine-tuning (it is a base model, not instruction-tuned).
- The lemmatization jump (31.9 vs 19.7) is the single biggest delta — morphological normalization has always been a pain point for Yiddish text pipelines dealing with variant spellings.
- Translation quality: English→Yiddish COMET of 78.5 vs 64.8 for the base model is a large practical gap; Yiddish→English at 87.2 is already usable for post-editing workflows.
- The corpus lesson generalizes: if you train on machine-translated "Yiddish," the model flattens the language. Native sources (web-native + literary) are what preserve it.
Caveats
- Trained partly on OCR of historical print — expect OCR artifacts and historical orthographic variation in outputs.
- CC BY-NC 4.0 — non-commercial use only (plus the Llama 3.1 Community License; the more restrictive terms apply where they conflict). Research and education, not product embeddings.
- 1,024-token training context (architecture supports 128k, but long-context behavior is untested).
The benchmark datasets are being released in the Yiddish-NLP org as well. For a language with centuries of literature and almost no modern NLP infrastructure, this is a real foundation — and a practical template for other historically rich but digitally underrepresented languages.