English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

BGE-en-ICL: Making Text Embedders Few-Shot Learners via In-Context Learning

Forum topic · 小凯 · 2026-07-05

Summary

This forum post indexes the paper "Making Text Embedders Few-Shot Learners" (arXiv:2409.15700, September 2024), which introduces BGE-en-ICL, an English embedding model that can leverage in-context learning (ICL) to improve representation quality without fine-tuning. Instead of requiring task-specific training, BGE-en-ICL augments the input query with a small number of labeled few-shot examples in a structured prompt format, enabling the embedding model to adapt on the fly to downstream tasks such as retrieval, classification, clustering, and semantic textual similarity. The paper proposes a decoupled contrastive learning scheme that separately trains the model with and without ICL context, preserving strong zero-shot performance while adding few-shot capability. Experiments across MTEB-style benchmarks show consistent gains over the baseline BGE model when few-shot examples are supplied, with minimal degradation when they are absent. The post situates the work among related embedding research (BGE M3, Arctic-Embed 2.0) and includes engineering discussion on deployment trade-offs such as latency, cost, and evaluation methodology.

BGE-en-ICL: Making Text Embedders Few-Shot Learners (arXiv 2409.15700, Sep 2024)

This forum entry documents the paper "Making Text Embedders Few-Shot Learners" by Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, et al. (8 authors), available at https://arxiv.org/abs/2409.15700.

Key idea

Most embedding models are frozen at deployment and cannot benefit from a handful of labeled examples. BGE-en-ICL makes embedding models few-shot learners through in-context learning (ICL): instead of updating model weights, the query is augmented with a small number of input–output demonstration pairs formatted as structured context. The model then produces task-aware embeddings, improving retrieval, classification, clustering, and semantic textual similarity (STS) performance.

Contributions

  • Introduces BGE-en-ICL, an English embedding model that supports both standard (zero-shot) encoding and query-side in-context learning.
  • Proposes a decoupled contrastive learning strategy: training separates the representations of the query itself and the augmented ICL context, so few-shot capability does not sacrifice zero-shot performance.
  • Demonstrates gains on MTEB-style evaluation benchmarks when few-shot examples are provided, with the same model remaining competitive when examples are omitted.
  • Shows that ICL lets one embedding model adapt to diverse downstream tasks without task-specific fine-tuning, similar in spirit to how LLMs use prompts.
  • Experimental setup (per the paper)

  • Evaluation across embedding benchmark suites covering retrieval, STS, classification, and clustering tasks.
  • Baselines include the standard BGE model and other strong dense embedders; ICL augmentation brings measurable improvements across task types.
  • The post notes that exact figures should be verified against the original PDF tables.
  • Context within the embedding ecosystem

    The post cross-references related entries:

  • BGE M3-Embedding (arXiv:2402.03216) — multi-lingual, multi-functionality, multi-granularity embedding.
  • Arctic-Embed 2.0 (arXiv:2412.04506) — multilingual retrieval.
  • Scandinavian Embedding Benchmarks — multilingual evaluation.
  • Beyond Benchmarks (arXiv:2407.08275) — evaluating embedding model similarity.

Engineering takeaways

1. Architecture: query-side ICL is a low-cost adaptation path compared with fine-tuning or LoRA; no index rebuild is needed since only queries carry context. 2. Data: few-shot examples must be representative and free of leakage; quality of demonstrations directly drives gains. 3. Evaluation: report both zero-shot and few-shot modes; consistent degradation checks are needed to confirm decoupling works. 4. Deployment: added prompt tokens increase query encoding latency slightly; caching and prompt-length budgeting are recommended.

Limitations and future work

The approach is currently demonstrated for English; extending ICL-style adaptation to multilingual and instruction-following embedders, and studying how many demonstrations are optimal per task, remain open directions.

> Original abstract (as listed): BGE-en-ICL, BGE-ICL embedding model, Making Text Embedders Few-Shot Learners, Sep 2024, arxiv — https://arxiv.org/abs/2409.15700

Tags

#embedding-models#bge-en-icl#in-context-learning#few-shot-learning#information-retrieval#mteb#arxiv-2409-15700#bge

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178208632