Summary
MMTEB (Massive Multilingual Text Embedding Benchmark), released in February 2025 by the Hugging Face team, is a large-scale extension of the MTEB benchmark covering 1,043 languages. The benchmark includes 550 tasks spanning bitext mining (the largest share), classification (255), clustering (209), and retrieval (142), covering applications from sentiment analysis and question-answering reranking to long-document retrieval. Tasks span 17 domains including legal, religious, programming, web, social, medical, blog, and academic text. The collection is subdivided into specialized benchmarks such as MTEB(eng, v2), MTEB(Multilingual, v1), and MTEB(Law, v1). Notably, the new MTEB(eng, v2) is much smaller and faster than the original English MTEB, making model submissions significantly cheaper and simpler. The benchmark is hosted on Hugging Face with a public leaderboard. Source paper: https://huggingface.co/papers/2502.13595. Per Tom Aarsen's announcement on LinkedIn.
MMTEB: Massive Multilingual Text Embedding Benchmark (February 2025)
MMTEB (Massive Multilingual Text Embedding Benchmark) is a major expansion of the MTEB embedding benchmark, released in February 2025 and hosted on Hugging Face with a public leaderboard.
Key Facts
- Language coverage: 1,043 languages in total
- Task count: 550 tasks
- Task breakdown:
- Primarily bitext mining (text pairing)
- 255 classification tasks
- 209 clustering tasks
- 142 retrieval tasks
- Task types: Sentiment analysis, question-answering reranking, long-document retrieval, and more
- Domains: 17 domains, including legal, religious, programming, web, social, medical, blog, and academic
Benchmark Subsets
The task collection is subdivided into separate benchmarks, such as:
- MTEB(eng, v2) — English benchmark
- MTEB(Multilingual, v1)
- MTEB(Law, v1)
The new MTEB(eng, v2) is much smaller and faster than the original English MTEB, making leaderboard submissions significantly cheaper and simpler.
Resources
- Paper: https://huggingface.co/papers/2502.13595
- Leaderboard: available on Hugging Face
Source: brief shared by Tom Aarsen on LinkedIn.
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178208725