PSK at SemEval-2026 Task 9: Multilingual Polarization Detection with Ensemble Gemma Models and Synthetic Data Augmentation

Forum topic · 小凯 · 2026-05-08

Summary

This paper presents the PSK system for SemEval-2026 Task 9, a binary polarization detection task spanning 22 languages. The approach fine-tunes separate Gemma 3 models (12B and 27B parameters) per language using Low-Rank Adaptation (LoRA), augmented with synthetic data generated by GPT-4o-mini through three strategies: direct generation, paraphrasing, and contrastive pair creation, followed by multi-stage quality filtering including embedding-based deduplication. Per-language threshold tuning on development data yields 2-4% F1 gains without retraining. The final system uses weighted ensembles of 12B and 27B model predictions with per-language strategy selection, achieving a mean macro-F1 of 0.811 across all 22 languages, ranking 2nd overall with 1st-place finishes in 3 languages and top-3 in 8. Notably, alternative architectures (XLM-RoBERTa, Qwen3) that performed strongly on the development set suffered 30-50% F1 drops on the test set, underscoring the importance of generalization in multilingual NLP systems.

Paper Overview

Field: NLP
Author: Srikar Kashyap Pulipaka
Published: 2026-05-06
arXiv: 2605.05159

Abstract

We present our system for SemEval-2026 Task 9: Multilingual Polarization Detection, a binary classification task spanning 22 languages. Our approach fine-tunes separate Gemma 3 models (12B and 27B parameters) per language using Low-Rank Adaptation (LoRA), augmented with synthetic data generated by a large language model (LLM). We employ three synthetic data strategies (direct generation, paraphrasing, and contrastive pair creation) using GPT-4o-mini, with a multi-stage quality filtering pipeline including embedding-based deduplication. We find that per-language threshold tuning on the development set yields 2 to 4% F1 improvements without retraining. We also use weighted ensembles of 12B and 27B model predictions with per-language strategy selection. Our final system achieves a mean macro-F1 of 0.811 across all 22 languages, ranking 2nd overall of the participating teams, with 1st place finishes in 3 languages and top-3 in 8 languages. We also find that alternative architectures (XLM-RoBERTa, Qwen3) that showed strong development set performance suffered 30 to 50% F1 drops on the test set, highlighting the importance of generalization.

Key Takeaways

Per-language LoRA fine-tuning of Gemma 3 (12B and 27B) is an effective recipe for multilingual polarization detection across 22 languages.
LLM-generated synthetic data (GPT-4o-mini) with three strategies and embedding-based deduplication improves training signal.
Per-language threshold tuning on the dev set yields 2-4% F1 improvement without retraining.
Weighted ensembles of 12B and 27B predictions with per-language strategy selection achieve a mean macro-F1 of 0.811: 2nd place overall, 1st in 3 languages, top-3 in 8.
Strong dev performance with XLM-RoBERTa and Qwen3 did not transfer, dropping 30-50% F1 on test — a cautionary lesson on generalization.

---

*Auto-collected on 2026-05-08.*