Soro: A Lightweight Foundation Model and Chatbot for Tajik

Forum topic · 小凯 · 2026-05-29

Summary

Soro is a family of Tajik-specialized conversational large language models designed for real-world deployment under tight compute and connectivity constraints in Tajikistan. Built from open-weight Gemma 3 checkpoints, the models undergo Tajik-only continual pretraining on a curated 1.9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials, followed by supervised instruction tuning on 40K Tajik teacher-style examples. Because standard benchmarks cover Tajik poorly, the authors introduce a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school and university entrance-exam domains, open-sourced on Hugging Face. Across these benchmarks, Soro substantially outperforms same-size Gemma 3 baselines while retaining strong English performance. FP8 and INT4 quantized variants preserve most Tajik capability, supporting pilot deployments in Tajikistan's education sector and at-scale rollout in schools. The paper (arXiv 2605.27379) by Stanislav Liashkov, Haitz Sáez de Ocáriz Borde, Azizjon Azimi, and colleagues targets low-resource language NLP for education.

Paper Overview

Field: NLP Authors: Stanislav Liashkov, Haitz Sáez de Ocáriz Borde, Azizjon Azimi, et al. Published: 2026-05-28 arXiv: 2605.27379

Abstract

We present Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and connectivity constraints in Tajikistan. Starting from open-weight Gemma 3 checkpoints, we perform Tajik-only continual pretraining on a curated 1.9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials, followed by supervised instruction tuning on 40K Tajik teacher-style examples. To enable rigorous evaluation despite the limited coverage of Tajik in standard benchmarks, we introduce a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school- and university entrance-exam domains, and we open-source them on Hugging Face. Across these Tajik benchmarks, Soro substantially outperforms same-size Gemma 3 baselines while retaining strong English performance. FP8 and INT4 quantized variants retain most Tajik capability, supporting pilots with Tajikistan's education sector and at-scale deployment in schools.

Key Points

Base models: open-weight Gemma 3 checkpoints
Pretraining data: 1.9 billion tokens of Tajik text (filtered web text, PDF documents, curriculum-aligned educational materials)
Fine-tuning: supervised instruction tuning on 40K Tajik teacher-style examples
New benchmarks: Tajik evaluation suite covering general knowledge, linguistic competence, and school/university entrance-exam domains, open-sourced on Hugging Face
Results: Soro substantially outperforms same-size Gemma 3 baselines on Tajik benchmarks while keeping strong English performance
Deployment: FP8 and INT4 quantized models retain most Tajik capability, enabling use in schools under compute and connectivity constraints

--- *Originally collected on 2026-05-29*