Overview
Field: NLP Authors: Stanislav Liashkov, Haitz Sáez de Ocáriz Borde, Azizjon Azimi, et al. Published: 2026-05-28 arXiv: 2605.27379
Summary
This paper introduces Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and connectivity constraints in Tajikistan.
- Starting from open-weight Gemma 3 checkpoints, the team performs Tajik-only continual pretraining on a curated 1.9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials.
- Followed by supervised instruction tuning on 40K Tajik teacher-style examples.
- Given the limited coverage of Tajik in standard benchmarks, the authors build a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school/university entrance-exam domains, open-sourced on Hugging Face.
- Results: Soro substantially outperforms same-size Gemma 3 baselines on the Tajik benchmarks while retaining strong English performance.
- FP8 and INT4 quantized versions preserve most of the Tajik capability, supporting pilot deployments and large-scale rollout in Tajikistan's education sector.
Original Abstract (excerpt)
> We present Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and connectivity constraints in Tajikistan. Starting from open-weight Gemma 3 checkpoints, we perform Tajik-only continual pretraining on a curated 1.9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials, followed by supervised instruction tuning on 40K Tajik teacher-style examples. To enable rigorous evaluation despite the limited coverage of Tajik in standard benchmarks, we introduce a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school- and university entrance-exam domains, and we open-source them on Hugging Face.
--- *Auto-collected on 2026-05-29*