Embedding models for RAG: how to choose one before you index
The embedding model is the hardest part of a RAG pipeline to change later: its vectors fill your index, and a new model means embedding everything again. So choose it on the numbers that govern it — vector size, context window, price — and on a retrieval test with your own queries, not on a benchmark score alone. Evanesca serves two open-weight embedding models, run in France: Qwen3 Embedding 8B and BGE Multilingual Gemma2.
Published 6 Oct 2026 · checked on 20 Oct 2026
Why the choice is hard to undo
Retrieval-augmented generation works in two phases. At index time, you cut each document into chunks, turn each chunk into a vector with the embedding model, and store the vectors. At query time, you turn the question into a vector with the same model, fetch the nearest chunks, and hand them to a chat model as context.
Vectors from two different models live in different spaces, and usually have different lengths: they cannot be compared. Switching models therefore means embedding the whole corpus again and rebuilding the index — for a large knowledge base, a migration rather than a setting. That is why the choice deserves a test before you index, not after.
The three numbers that decide it
- Dimensions: the length of each vector. Longer vectors can hold more nuance, but every extra dimension is stored for every chunk and compared at every query.
- Context window: how many tokens one input may hold. A larger window lets you embed a long section as one vector instead of cutting it into small pieces.
- Price: embeddings are billed on input tokens only — a vector is not text, so there is no output price. Your indexing bill is corpus size times input price.
A benchmark score is useful at the margin, but it averages public datasets that do not know your documents, your language mix or your chunking. The three numbers above are the ones you will live with.
The two embedding models on Evanesca
| Model | Maker | Licence | Dimensions | Context (tokens) | Price | Run by |
|---|---|---|---|---|---|---|
| Qwen3 Embedding 8B | Qwen | Apache-2.0 | 4,096 | 40,960 | €0.168 | Scaleway, Paris |
| BGE Multilingual Gemma2 | BAAI | Gemma Terms of Use | 3,584 | 8,192 | €0.168 | Scaleway, Paris |
Dimensions were measured with a real call; the context window is the limit each operator announces. For Qwen3 Embedding 8B, Scaleway accepts 40,960 tokens per input, while the model card lists a 32k context. Licences differ: Qwen3 Embedding 8B is Apache-2.0, while BGE Multilingual Gemma2 comes under Google's Gemma Terms of Use — open weights, not an open-source licence. Every detail is on the embeddings page; the model cards quoted in this article were read on 6 October 2026.
A third model, BGE-M3 (MIT, 1,024 dimensions), is paused since 20 October 2026: OVHcloud, which ran it in Gravelines, no longer serves it for Evanesca. An index built with it has to be embedded again with one of the models above before it can be queried through Evanesca.
What the MTEB scores say, and what they do not
MTEB, the Massive Text Embedding Benchmark, averages a model over many public tasks. Qwen3 Embedding 8B and BGE-M3 (paused at Evanesca, see above) appear in the same table of the Qwen3 Embedding technical report, on MTEB Multilingual:
| Model | Size | Mean (task) | Retrieval |
|---|---|---|---|
| BGE-M3 | 0.6B | 59.56 | 54.60 |
| Qwen3 Embedding 8B | 8B | 70.58 | 70.88 |
| BGE Multilingual Gemma2 | — | not in that table | — |
- These are the Qwen team's figures, published in June 2025; the score it gives for BGE-M3 was taken from the public MTEB leaderboard on 4 June 2025. Evanesca has not re-run them, and leaderboard positions have moved since.
- For RAG, the Retrieval column matters most, and the gap there is about 16 points — wider than on the overall mean.
- BGE Multilingual Gemma2's card publishes results task by task but no aggregate comparable to this table, so no score is quoted for it here rather than a guess.
What longer vectors cost
At these prices, indexing is cheap; storage and search speed are where vector length shows. For a corpus of 100 million tokens and an index of one million chunks stored as 32-bit floats:
| Model | One vector | One million vectors | Embedding 100 million tokens |
|---|---|---|---|
| Qwen3 Embedding 8B | 16.4 KB | 16.4 GB | €16.80 |
| BGE Multilingual Gemma2 | 14.3 KB | 14.3 GB | €16.80 |
Qwen3 Embedding 8B's card says it can also produce shorter vectors, from 32 to 4,096 dimensions. Whether an operator's API lets you ask for a shorter length is something to test before you plan your storage around it.
Getting more from any model
- Cut chunks along meaning — sections, paragraphs — with a small overlap, rather than at a fixed length that splits sentences.
- Strip headers, menus and boilerplate before embedding: they pollute the nearest neighbours.
- Apply the same clean-up to queries as to documents, and always embed both with the same model.
- Store the model's name next to the index, so that vectors from two models are never mixed.
BGE-M3's card recommends hybrid retrieval — embeddings plus keyword matching such as BM25 — followed by a re-ranker. Evanesca's embeddings endpoint follows OpenAI's format and returns one dense vector per input, so the keyword half can come from your search engine.
How to choose before you index
- Measure your corpus: its size in tokens and its mix of languages.
- Take a few hundred real documents and real questions, and check, for each candidate, whether the right passage comes back in the first results.
- Cost it: corpus tokens times input price for the indexing, and vector length times chunk count for the storage.
- Choose, index, and stay with the model unless a test on your own data says otherwise.
If your documents are long, a larger context window buys you freedom in how you cut them; if your corpus is huge, shorter vectors keep the index lean. The answering half of the pipeline is a chat model: the guide to open-weight models helps choose it.
Documents are the sensitive part
Embedding sends whole documents through the model — often the most sensitive data in the pipeline. Evanesca's embedding models are run in France, at Scaleway in Paris — a French company with no US parent. They use the same key and the same prepaid credit as the chat models (API documentation), and Evanesca does not store what you send.
One exception sits with an operator: Scaleway, which runs Qwen3 Embedding 8B and BGE Multilingual Gemma2, may keep and read requests that cause errors or look abusive, for up to 14 days (Scaleway, read on 6 October 2026). Prices are on the pricing page: no subscription, and credit from €2.
Frequently asked questions
What is the best open-weight embedding model for RAG?
There is no single best: the choice trades vector size, context window and price against retrieval quality on your data. On MTEB Multilingual, as reported by the Qwen team in June 2025, Qwen3 Embedding 8B scores 70.58 on average against 59.56 for BGE-M3, which is paused at Evanesca. Of the two models Evanesca serves, Qwen3 Embedding 8B gives vectors 1.14 times as long as BGE Multilingual Gemma2's (4,096 against 3,584 dimensions), at the same price per token, and accepts longer inputs. Test both on your own queries.
Are embeddings billed on input or output tokens?
On input tokens only. An embedding call returns a vector, not text, so there is no output price: the cost of indexing is the number of tokens in your corpus times the input price.
Do I have to re-embed everything if I change model?
Yes. Vectors from two models are not comparable, and their lengths usually differ, so the corpus has to be embedded again and the index rebuilt. That is why the model is worth testing before you index a large corpus.
Can I embed queries with a different model than documents?
No. A query vector can only be compared with document vectors from the same model. Use one model for both sides, and record its name with the index.
Read next
- Self-hosting an LLM or paying per token: where the break-even really isA rented H100 costs the same whether it serves one token or a billion. The break-even against per-token prices, worked out with real October 2026 prices.
- AI API aggregators: one key for many models, and the question of who runs themAn AI API aggregator puts many models behind one OpenAI-compatible key. How OpenRouter, Together AI, Groq, Fireworks AI and Evanesca differ.
- How to choose an AI API aggregator for your companyFive questions to ask an AI API aggregator before you sign — jurisdiction, data kept, billing, catalogue, compatibility — and how five providers answer them.