Releasing Bekko Embedding, an Ultra-Compact Multilingual Text Retrieval Model

I released Bekko Embedding, a family of ultra-compact multilingual text retrieval models. Despite having only 8M and 25M active parameters, they achieve retrieval performance comparable to models several to more than ten times larger. I think they are especially well suited to use cases that need to run vector retrieval on a CPU.

The term active parameters (AP) can be confusing. Here, I use it to mean roughly the parameter count excluding the token embedding lookup table. These are not Mixture-of-Experts models; they are simple dense models.

The following article was originally published on the Hugging Face blog as Bekko Embedding: how small can a multilingual retrieval model be?.

Text embedding models have been getting better and bigger at the same time. In the multilingual range from 100M to 600M total parameters, the options include EmbeddingGemma, Granite Embedding R2 311M, gte-multilingual-base, BGE-M3 and Arctic Embed. Above them sits a newer class built on LLM backbones, several billion parameters and up, with models such as Nemotron-3-Embed-8B, Qwen3-Embedding-8B and pplx-embed-v1-4b. If you have a GPU setup that runs them fast enough, those larger, stronger models are the ones to reach for.

What interested me, though, was the opposite of that push for scale and quality — the approach of making the model as small as possible. How far down can you go while keeping quality that is actually usable for real retrieval? Bekko Embedding is where I landed — two models, bekko-embedding-v1-a8m and bekko-embedding-v1-a25m, released under the MIT license and shortened below to bekko-a8m and bekko-a25m.

The smaller one has just 7.67M active parameters. Active parameters are the weights used in the transformer computation for every token, excluding the token embedding lookup table. On the 18 retrieval tasks of the official MMTEB Multilingual v2 suite, bekko-a8m averages 56.2, ahead of every multilingual-e5 model and BGE-M3, which have roughly 3x to 40x its active parameters. The larger bekko-a25m scores higher still.

You can try them without installing anything: the bekko-embedding-web demo loads the model into your browser and runs it there.

Why I care about the small end

Coding agents shifted where the compute happens. The reasoning runs on a frontier model behind an API, so the machine in front of you matters less for that part. What still runs locally is everything the agent needs in order to find things: embedding documents into vectors, building the index, searching through context.

That local machine usually has no GPU, and it can be as modest as a Raspberry Pi. Embedding ten thousand documents with multilingual-e5-large takes about eight minutes on a fast desktop CPU and closer to two hours on a Raspberry Pi 5. And query embeddings are computed on every single search, not once at index time.

Small embedding models — say, under 30M active parameters — are also thin on the ground. Among the openly licensed multilingual contextual models I surveyed, only two besides Bekko sit below 30M active parameters with usable retrieval quality: multilingual-e5-small, initialised from a MiniLM distilled back in 2020, and granite-embedding-97m-r2, released in 2026 at 28.3M active parameters. That is a snapshot of what I looked at rather than an exhaustive registry, and it will date as new models appear.

Static embeddings such as Model2Vec do go smaller. They are not contextual models in the way a transformer is, though, and quality drops accordingly.

Model parameters and compute

Multilingual embedding models look bigger than they are. A model card says 118M parameters, and it is easy to assume that costs 118M parameters' worth of compute. But in a multilingual encoder most of that number is the token embedding table for all those languages (256,000 tokens in Bekko's case), and at inference that table is only ever looked up: one entry per token, no matrix multiplication, just a read.

The parameters that actually get multiplied for every token are the ones in the transformer blocks. Following the convention used for scaling-law parameter counts, I call those the active parameters (AP). They are a far more useful number than the total parameter count when you are reasoning about inference cost.

The term is overloaded, so one clarification. This is not the Mixture-of-Experts sense of the phrase, where a router activates a different subset of experts for each token and the active count is a fraction of the weights that exist. Bekko is a dense encoder: all 7.67M parameters run at inference for every token, and the only thing the count leaves out is the embedding lookup.

multilingual-e5-small, for example, has 118M total parameters, of which 21.6M are active. bekko-a8m has 106M total, of which 7.67M are active. That gap is what shows up in inference time. Active parameters alone do not determine throughput, but smaller generally means faster.

Bekko Embedding

Bekko is built by pruning the 22-layer mmBERT-small encoder down to 4 layers (a8m) and 13 layers (a25m), then training the two pruned models on about 1.1 billion multilingual pairs. There is more on that at the end; what matters first is what the models do.

• 100+ languages in the training data, cross-lingual by design: a Japanese query can retrieve an English document without translation. The per-language results below cover 14 of them.
• 8,192-token inputs on both models.
• 384 dimensions, truncatable to 256 / 128 / 64 thanks to Matryoshka training. Truncating bekko-a25m to 256 dimensions costs about 1.3% relative retrieval quality on HAKARI-Bench Overall, and 128 dimensions about 6.2%.
• MIT-licensed weights.
• The data used for training is public too: the stage-1 corpus and the mined hard negatives.

Benchmark results

The figure below has six columns. The first two are two readings of the same MMTEB Multilingual v2 run: ALL is the mean over all 131 tasks, Ret is the mean over its 18 retrieval tasks alone. Competitor scores come from the official public snapshot, and Bekko was evaluated over the same task set with the same aggregation rules.

The other four columns are retrieval-only benchmarks. Multilingual NanoBEIR is the original English NanoBEIR plus 13 community-published translations, none of them mine. The other Nano sets are ones I built for HAKARI-Bench, described at the end of this paragraph. NanoRTEB is a compact version of RTEB, built from production-style retrieval sets. NanoCoIR covers code retrieval and NanoLongEmbed covers long inputs. All four run in one harness with identical settings, HAKARI-Bench (arXiv), which is also mine, and BM25 is included as a lexical baseline.

The two MMTEB columns are worth reading against each other, because they disagree. granite-embedding-97m-r2 is the clearest case: 60.3 on the 18 retrieval tasks, the fourth-best in the comparison, and 51.9 over all 131, the lowest. Training hard on retrieval tasks can trade away the rest of the suite. Bekko sits in between. Retrieval is what I optimised for, but the all-task mean holds up well enough.

bekko-a8m scores higher than multilingual-e5-small in all six columns, and multilingual-e5-small carries 2.8x — nearly triple — its active parameters. bekko-a25m is ahead of multilingual-e5-large and BGE-M3 on the retrieval column with a twelfth of their active parameters, though both are stronger over the full 131 MMTEB tasks.

But the comparison is not a clean sweep. Five models beat bekko-a25m on the 18 retrieval tasks: harrier-oss-v1-270m, granite-embedding-311m-r2, EmbeddingGemma-300M, granite-embedding-97m-r2 and snowflake-arctic-embed-l-v2.0. Four beat it on the all-task mean: harrier-oss-v1-270m and EmbeddingGemma-300M again, plus BGE-M3 and multilingual-e5-large, and gte-multilingual-base is level.

Every one of them except granite-embedding-97m-r2 carries 4x to 13x bekko-a25m's active parameters. If retrieval accuracy is what you optimise for and you have the compute, they are excellent models.

Code retrieval is a mixed result: bekko-a25m reaches 78.6, ahead of multilingual-e5-large and BGE-M3 but behind EmbeddingGemma-300M at 84.7 and granite-embedding-311m-r2 at 81.4. Long inputs are more surprising. BM25 wins outright at 82.2, since lexical matching is hard to beat when you are looking for one passage inside a long document, but among the twelve dense models bekko-a25m ranks first at 70.6, with bekko-a8m third.

Languages

Averages can hide a language where a model falls apart, so here is each of the 14 languages in Multilingual NanoBEIR separately.

bekko-a25m scores above multilingual-e5-small and BM25 in all 14 languages, and its largest deficit to BGE-M3 in any single language is 2.7 points. Thai, Arabic and Serbian are Bekko's weakest of the 14 and have the most room to improve.

Speed

Active parameter count is a good first-order predictor of throughput. I measured six models under identical settings on four machines, from a Raspberry Pi 5 to an RTX 5090:

bekko-a8m is the fastest of the six on every one of them. Real indexing jobs mostly run on a desktop CPU: turning 100,000 documents into vectors on the Ryzen 9 7950X takes about 5 minutes with bekko-a8m and about 80 minutes with multilingual-e5-large. The Raspberry Pi 5 is the one I find more interesting. 33 documents per second is glacial by GPU standards, but it turns a 10,000-document index into a five-minute job — fast enough to be practical, depending on what you are doing — against the two hours the same board needs for multilingual-e5-large.

Both CPU panels run each model through the OpenVINO backend, using each model's own published OpenVINO export. It was the fastest CPU backend for all six, and running some of them without OpenVINO would not make a fair comparison. These numbers should not be assumed for a model that has no equivalent export. Bekko ships OpenVINO builds for both sizes.

Active parameters do not explain every throughput result. multilingual-e5-small has 87% of bekko-a25m's active parameters but 1.7x its CPU throughput, and granite-embedding-97m-r2 has 14% more active parameters at almost exactly the same throughput. My reading is that this is architectural. The ModernBERT architecture's efficiency work (unpadding, variable-length FlashAttention kernels, alternating global/local attention) is explicitly designed around GPU kernels, and a plain XLM-R-style encoder architecture seems to map more cleanly onto CPU backends. I have not gone through the implementations to confirm this, so treat it as my interpretation rather than a measured result. Even so, bekko-a8m is small enough that it comes out ahead regardless.

124 MiB, and it runs in a browser

The vocabulary table that inflates the parameter count also inflates the download. Since it is a static lookup, the ONNX and OpenVINO builds store it as row-wise int8 while keeping the transformer weights in fp32. In the release checks the minimum cosine similarity against the PyTorch model stays at 0.9994 or above.

That brings bekko-a8m to a 124 MiB artifact and bekko-a25m to 190 MiB, small enough to send to a browser. The transformer weights could be taken from float down to int8 as well for a smaller download, but the quality loss was severe, so I left them alone.

Same Natural Questions inputs as the speed figure above; the last row is the same model running natively, as a reference point rather than a browser result.

A browser demo is at huggingface.co/spaces/hotchpotch/bekko-embedding-web. The embedding runs client-side, so your text is not sent to a server to be encoded.

Where another model may fit better

• If you want the best retrieval accuracy in this comparison and have the compute, harrier-oss-v1-270m, granite-embedding-311m-r2 and EmbeddingGemma-300M score higher.
• If you need a general-purpose sentence encoder, Bekko's classification and clustering scores are mid-table to weak. For anything that is not retrieval, another model is often the better fit.

Getting started

If you are not sure which size to start with: a8m for CPU indexing and in-browser retrieval, a25m when you have a GPU or Apple MPS available and retrieval quality matters more than throughput.

You can run either with plain PyTorch, with OpenVINO on CPU, or with Transformers.js in a browser. The same model ID works in all three and none of them needs prefixes, though each runtime has its own API.

pip install -U "sentence-transformers>=5.0" "transformers>=5.12"

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("hotchpotch/bekko-embedding-v1-a8m")

query = "What are the characteristics of sushi?"
docs = [
    "A warm noodle soup served in broth with sliced toppings.",
    "天ぷらは魚や野菜に衣をつけて揚げた料理です。",
    "Une fine crepe garnie de sucre, de beurre ou de fruits.",
    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
]

scores = util.cos_sim(
    model.encode(query, normalize_embeddings=True),
    model.encode(docs, normalize_embeddings=True),
)[0]
print("best doc:", docs[int(scores.argmax())])

No prefixes or task instructions: queries and documents use the same call. For a8m under the conditions above, the OpenVINO backend is about 2.8x faster on x86 than PyTorch and 1.7x faster on a Raspberry Pi 5.

pip install -U \
  "sentence-transformers[openvino]>=5.0" \
  "transformers>=4.57,<5"

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer(
    "hotchpotch/bekko-embedding-v1-a8m",
    backend="openvino",
    device="cpu",
    model_kwargs={"file_name": "openvino/openvino_model.xml", "device": "CPU"},
)

query = "What are the characteristics of sushi?"
docs = [
    "A warm noodle soup served in broth with sliced toppings.",
    "A Japanese dish made with vinegared rice, often shaped with seafood, vegetables, or egg.",
]

scores = util.cos_sim(
    model.encode(query, normalize_embeddings=True),
    model.encode(docs, normalize_embeddings=True),
)[0]
print("best doc:", docs[int(scores.argmax())])

Transformers.js runs the same model client-side, with WebGPU where the browser has it.

npm install @huggingface/transformers

import { pipeline } from "@huggingface/transformers";

// Browser: use WebGPU when available, otherwise fall back to WASM.
// Node.js: replace this line with `const device = "cpu";`.
const device = navigator.gpu ? "webgpu" : "wasm";

const extractor = await pipeline(
  "feature-extraction",
  "hotchpotch/bekko-embedding-v1-a8m",
  { device, dtype: "fp32" },
);

const embedding = await extractor("What are the characteristics of sushi?", {
  pooling: "mean",
  normalize: true,
});

console.log(embedding.dims); // [1, 384]

Links:

• Models: bekko-embedding-v1-a8m · bekko-embedding-v1-a25m
• Browser demo: bekko-embedding-web
• Training data: stage-1 corpus · hard negatives
• Paper: arXiv:2607.25180

How it was trained

Briefly, since this post is about using the models. I pruned mmBERT-small from 22 layers to 4 and 13, keeping the early layers plus one deep global-attention layer, and used the pruned checkpoints directly as base models. Then two stages of contrastive learning: about 1.1 billion multilingual pairs from the corpus I published, which leans heavily on synthetic data and includes two complementary families of LLM-synthesized queries, followed by hard-negative fine-tuning that includes 8,192-token long-document negatives. Neither stage uses a teacher model or distillation.

Both models were trained on a single GPU, an RTX PRO 6000 Blackwell Max-Q: about three days for a8m and about eight days for a25m. Multilingual embedding models at the usual scale are trained on clusters, and BGE-M3 reports up to 96 GPUs for its training. A small active-parameter budget is not only cheaper to run at inference time; it also brings training back to one workstation GPU. An RTX 5090 works too, at the cost of a longer run. Not needing a fleet of GPUs is a large part of why I find models this size worth working on.

The work leans heavily on others: mmBERT and ModernBERT for the backbone, E5, GTE, BGE-M3, Qwen3 Embedding and Ruri for the training recipe, and sentence-transformers for the implementation. If you are interested in how a model this small can carry multilingual retrieval quality, the paper has the layer-selection study, the loss, how the 1.1 billion pairs were assembled and synthesized, and the full evaluation: Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders.

If you are looking for a small multilingual retrieval model, I would be glad if Bekko Embedding turns out to be a good option for you.