Embedding model shootout: accuracy vs. speed on CPU
A head-to-head comparison of six open embedding models on retrieval accuracy, latency, and memory footprint — all measured on the same CPU-only machine.
Choosing an embedding model is more consequential than most RAG guides admit.
Vector similarity is only as meaningful as the embedding space it lives in. We ran six open-weight embedding models through a standardized retrieval benchmark — BEIR-subset, 1M candidate pool — on a CPU-only machine to remove GPU variance from the picture.
The results are surprising. The largest model is not always the most accurate on domain-specific queries. A 384-dimension model from 2022 outperforms a 1536-dimension model from 2024 on technical question-answering, at one-third the latency.
We report accuracy (NDCG@10), median and P99 latency, peak RAM, and the model versions used. Everything is reproducible with the published Dockerfile.
Keep your edge
Join The Signal for more on human capability in the age of AI.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.