Task-Adaptive Embedding Refinement via Test-time LLM Guidance

Forum topic · 小凯 · 2026-05-14

Summary

This paper explores an LLM-guided query refinement paradigm that extends the usability of text embedding models to challenging zero-shot search and classification tasks. The method refines a user query's embedding representation using feedback from a generative LLM on a small set of documents, allowing embeddings to adapt in real time to the target task without additional training. Extensive experiments with state-of-the-text embedding models across diverse search and classification benchmarks show consistent gains across all models and datasets, with relative improvements of up to +25% in literature search, intent detection, key-point matching, and nuanced query-instruction following. The refined queries improve ranking quality and induce clearer binary separation over the corpus, making the embedding space better reflect task-specific constraints of each ad-hoc query. Authored by Ariel Gera, Shir Ashury-Tahan, Gal Bloch, Ohad Eytan, and Assaf Toledo, the paper is available at arXiv:2605.12487.

Paper Overview

Research area: NLP
Authors: Ariel Gera, Shir Ashury-Tahan, Gal Bloch, Ohad Eytan, Assaf Toledo
Published: 2026-05-12
arXiv: 2605.12487

Abstract

We explore the effectiveness of an LLM-guided query refinement paradigm for extending the usability of embedding models to challenging zero-shot search and classification tasks. Our approach refines the embedding representation of a user query using feedback from a generative LLM on a small set of documents, enabling embeddings to adapt in real time to the target task. We conduct extensive experiments with state-of-the-art text embedding models across a diverse set of challenging search and classification benchmarks. Empirical results indicate that LLM-guided query refinement yields consistent gains across all models and datasets, with relative improvements of up to +25% in literature search, intent detection, key-point matching, and nuanced query-instruction following. The refined queries improve ranking quality and induce cleaner binary separation over the corpus, so that the embedding space better reflects the nuanced, task-specific constraints of each ad-hoc user query.

Highlights

Training-free, test-time adaptation of query embeddings via LLM feedback on a few documents.
Works across state-of-the-art embedding models with consistent gains on all evaluated benchmarks.
Up to +25% relative improvement in literature search, intent detection, key-point matching, and query-instruction following.

---

*Auto-collected on 2026-05-14.*