Paper Overview
Field: NLP Authors: Stanislav Liashkov, Haitz Sáez de Ocáriz Borde, Azizjon Azimi, et al. Published: 2026-05-28 arXiv: 2605.27379
Summary
This paper presents Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and connectivity constraints in Tajikistan. Starting from open-weight Gemma 3 checkpoints, the team performs Tajik-only continual pretraining on a curated 1.9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials, followed by supervised instruction tuning on 40K Tajik teacher-style examples.
To enable rigorous evaluation despite the limited coverage of Tajik in standard benchmarks, the authors introduce a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school- and university entrance-exam domains, and open-source them on Hugging Face. Across these Tajik benchmarks, Soro substantially outperforms same-size Gemma 3 baselines while retaining strong English performance. FP8 and INT4 quantized variants retain most of the models' Tajik capability, supporting pilots in Tajikistan's education sector and scaled deployment in schools.
Key Contributions
- Continual pretraining: Tajik-only adaptation of Gemma 3 on 1.9B tokens (filtered web text, PDFs, curriculum-aligned educational materials)
- Instruction tuning: Supervised fine-tuning on 40K Tajik teacher-style examples
- New benchmarks: A Tajik evaluation suite (general knowledge, linguistic competence, school/university entrance-exam domains), open-sourced on Hugging Face
- Deployment focus: FP8 and INT4 quantization preserving most Tajik capability for resource-constrained environments
Results
Soro substantially outperforms same-size Gemma 3 baselines across the new Tajik benchmarks while maintaining strong English performance.
--- *Auto-collected on 2026-05-29*