# Qwen3-Embedding-8B Review — Full Performance Analysis & Comparison ## Overview **Qwen3-Embedding-8B** is the flagship 8-billion-parameter text embedding model from Alibaba's Qwen team, released June 2025. It is an open-weight model (Tongyi Qianwen custom commercial license) that supports **100+ languages**, a **32K token context window**, and **configurable embedding dimensions (32–4096)** via Matryoshka Representation Learning (MRL). --- ## MTEB Benchmarks — The Key Numbers ### 🏆 MTEB Multilingual Leaderboard (as of June 2025 — #1 position) | Model | Size | Mean Score (Tasks) | Mean Score (Type) | Key Strengths | |---|---|---|---|---| | **Qwen3-Embedding-8B** | **8B** | **70.58** | **61.69** | Bitext Mining (80.89), Classification (74.00), Pair Classification (86.40), Retrieval (70.88) | | Qwen3-Embedding-4B | 4B | 69.45 | 60.86 | Instruction Retrieval (11.56), Retrieval (69.60) | | Qwen3-Embedding-0.6B | 0.6B | 64.33 | 56.00 | Efficient for resource-constrained scenarios | ### MTEB v2 Open-Weight Leaderboard (Presenc AI, May 2026) | Rank | Model | Parameters | Avg MTEB Score | |---|---|---|---| | **1** | **Qwen3-Embedding-8B** | **~8B** | **~75.1** | | 2 | Qwen3-Embedding-4B | ~4B | ~72.4 | | 3 | NV-Embed-v2 | ~7B | ~72.3 | | 4 | GTE-Qwen2-7B-Instruct | ~7B | ~70.2 | | 5 | Stella-en-1.5B-v5 | ~1.5B | ~69.4 | | 6 | BGE-M3 | ~0.6B | ~68.2 | | 7 | Linq-Embed-Mistral | ~7B | ~68.2 | | 8 | SFR-Embedding-Mistral | ~7B | ~67.6 | | 9 | Qwen3-Embedding-0.6B | ~0.6B | ~67.8 | | 10 | E5-Mistral-7B-Instruct | ~7B | ~66.6 | | 11 | jina-embeddings-v3 | ~0.6B | ~65.5 | | 12 | BGE-Large-EN-v1.5 | ~0.3B | ~64.2 | | 13 | mxbai-embed-large-v1 | ~0.3B | ~64.7 | | 14 | Nomic Embed Text v2 | ~0.5B | ~63.8 | --- ## Comparison with Key Competitors ### 🆚 Qwen3-Embedding-8B vs BGE-M3 (BAAI) | Feature | Qwen3-Embedding-8B | BGE-M3 | |---|---|---| | **Parameters** | 8B | 0.6B | | **MTEB Avg Score** | ~75.1 | ~68.2 | | **Languages** | 100+ (strong Chinese/Asian) | ~100 | | **Context Length** | 32K | 8K | | **Retrieval** | Dense + multi-vector | Dense + sparse + multi-vector | | **License** | Custom (commercial) | MIT (permissive) | | **Self-hosted Cost** | ~$0.005/M tokens | ~$0.002/M tokens | | **HuggingFace Popularity** | Fast-growing adoption | Most downloaded (tens of millions) | **Bottom line**: Qwen3 is significantly more accurate (+7 pts MTEB) but larger and under a less permissive license. BGE-M3 is the workhorse for lightweight, permissive deployments. ### 🆚 Qwen3-Embedding-8B vs OpenAI text-embedding-3-large | Feature | Qwen3-Embedding-8B (self-hosted) | OpenAI text-embedding-3-large | |---|---|---| | **MTEB Score** | ~75.1 | ~64.0 (reported across benchmarks) | | **Dimensions** | 32–4096 (configurable) | 256–3072 (configurable) | | **Languages** | 100+ | 50+ | | **Cost per 1M tokens** | ~$0.005 (self-hosted) | $0.13 | | **Context** | 32K | 8K | **Bottom line**: Qwen3 surpasses OpenAI's offerings on MTEB scores, offers longer context, and at scale costs ~1/26th the price when self-hosted. ### 🆚 Qwen3-Embedding-8B vs NV-Embed-v2 (NVIDIA) | Feature | Qwen3-Embedding-8B | NV-Embed-v2 | |---|---|---| | **MTEB Score** | ~75.1 | ~72.3 | | **Languages** | 100+ (multilingual) | English-focused | | **License** | Custom commercial | CC-BY-NC | | **Parameters** | 8B | ~7B | **Bottom line**: Qwen3 leads in both multilingual coverage and overall MTEB score. NV-Embed-v2 is a strong English-only alternative. ### 🆚 Qwen3-Embedding-8B vs GTE-Qwen2-7B-Instruct | Feature | Qwen3-Embedding-8B | GTE-Qwen2-7B-Instruct | |---|---|---| | **MTEB Score** | ~75.1 | ~70.2 | | **License** | Custom commercial | Apache 2.0 | | **Code Retrieval** | Strong | Particularly strong (code-focused) | **Bottom line**: Qwen3 has higher overall scores. GTE-Qwen2 wins on Apache 2.0 license and code retrieval. --- ## Closed API Comparison The **Presenc AI report (May 2026)** compares Qwen3 against closed APIs: | Model | MTEB Score | Cost per 1M tokens | |---|---|---| | Voyage AI voyage-3 | ~74.0 | $0.06 | | **Qwen3-Embedding-8B (self-hosted)** | **~75.1** | **~$0.005** | | Cohere Embed v3 English | ~64.0 | $0.10 | | OpenAI text-embedding-3-small | ~62.3 | $0.02 | | Google Gemini Embedding 001 (English leaderboard) | 68.32 (English) | Varies | **Key insight**: Qwen3-Embedding-8B matches or exceeds all closed APIs on quality while offering drastically lower cost at scale. --- ## Agentset Live Performance Data From **Agentset.ai** evaluations (830 evaluations logged): - **ELO Rating**: 1510 (#8 of 18 on leaderboard) - **Accuracy (nDCG@10)**: 0.718 (#1 ranked) - **Win Rate**: 48.8% - **Latency**: ~41ms (mean) - **Price**: $0.05 per 1M tokens (via API) | Dataset | ELO | Win Rate | nDCG@10 | Latency | |---|---|---|---|---| | MSMARCO | 1500 | 57.5% | 0.937 | 39ms | | FiQa | 1500 | 50.7% | 0.880 | 30ms | | DBPedia | 1500 | 58.1% | 0.797 | 49ms | | SciFact | 1500 | 41.3% | 0.744 | 41ms | | ARC | 1500 | 52.5% | 0.857 | 35ms | --- ## Strengths ✅ 1. **#1 on MTEB Multilingual Leaderboard** — 70.58 score, the highest among all open-weight embedding models 2. **Excellent multilingual capabilities** — Supports 100+ languages with particularly strong Asian language performance 3. **32K token context** — 4x longer than OpenAI and most competitors (8K) 4. **Flexible dimensions (32–4096)** via MRL — great for storage optimization 5. **Cost-effective at scale** — ~$0.005/M tokens self-hosted vs $0.13 for OpenAI 6. **Strong Reranker counterpart** — Qwen3-Reranker-8B scores 69.02 on MTEB-RC ## Weaknesses ❌ 1. **Custom commercial license** — Not fully open-source (Apache 2.0 or MIT) like BGE-M3 or GTE-Qwen2 2. **Large model (8B)** — Requires significant GPU memory (needs ~16GB+ VRAM), not suitable for edge/on-device 3. **English-only alternatives outperform** — For English-only tasks, NV-Embed-v2 or Stella may be more efficient 4. **Higher latency than smaller models** — ~41ms vs sub-10ms for BGE-M3 ## When to Use Qwen3-Embedding-8B | Use Case | Recommendation | |---|---| | 🌍 **Multilingual RAG** (global applications) | **Best choice** — #1 multilingual MTEB | | 🏢 **Enterprise production RAG** (quality-focused) | **Excellent** — highest accuracy open model | | 💰 **Cost-sensitive at scale** | **Great** — self-hosting is ~26x cheaper than OpenAI | | 📱 **Edge/on-device deployment** | ❌ Too large — use Qwen3-0.6B or BGE-M3 instead | | 📄 **Long document RAG** (32K chunks) | **Best choice** — longest context window | | 🇬🇧 **English-only applications** | Overkill — consider NV-Embed-v2 or Stella | | ⚖️ **Max license freedom** | ❌ BGE-M3 (MIT) or GTE-Qwen2 (Apache 2.0) are better | --- ## Verdict **Qwen3-Embedding-8B is currently the highest-performing open-weight embedding model** on the MTEB multilingual leaderboard, surpassing both other open models (NV-Embed-v2, GTE-Qwen2, BGE-M3, E5-Mistral) and proprietary APIs (OpenAI, Cohere) on overall quality. Its 32K context window, flexible dimensions via MRL, and strong multilingual support make it ideal for production RAG pipelines — especially global applications. The main trade-offs are its larger size (requiring GPU infrastructure), custom commercial license (not fully open-source), and the fact that smaller, more permissive models like BGE-M3 may be better for lightweight or permissive-license deployments.