Lexical search is still effective and cheap. It does fail on one specific class of queries, and understanding which one keeps you from replacing it where it works perfectly well.
What an inverted index actually does
An inverted index maps each term to the list of documents containing it. A weighting function such as BM25 then ranks documents by the frequency of the query terms, damped by how common the term is across the corpus and by document length. The model is a bag of words: term order plays no part, and two distinct terms are two orthogonal dimensions.
That property is also the limit. “Electric bike” and “e-bike” share no token once tokenized apart. A lexical index treats them as unrelated, however good the weighting. The classic remedies — synonym lists, stemming, query expansion — are hand-maintained patches that age badly and multiply as soon as you add a language.
What dense representation brings
An embedding model projects text into a fixed-dimension space, typically a few hundred components, where geometric proximity approximates proximity of meaning. Two vectors are then compared with cosine similarity:
cos(a, b) = (a · b) / (‖a‖ · ‖b‖)Cosine ignores magnitude and keeps only direction, which makes it insensitive to the length of the encoded text. If vectors are normalized at production time, cosine reduces to the dot product, which is cheaper to evaluate — that is why most indexes normalize at ingestion.
The practical payoff: two different phrasings of the same need land close together in the space, with no synonym dictionary. With a multilingual model, a French query can reach an English document, because both are projected into the same space.
Where lexical still wins
Dense search behaves poorly on exact identifiers. A product reference, a serial number, a rare proper noun or a code snippet is often absent from — or poorly represented in — the model’s vocabulary. The inverted index finds them exactly. Negation and hard constraints escape it too: an embedding does not represent “gluten free” in a reliably opposite way to “with gluten”.
That is why production systems combine both signals. You run both searches in parallel and merge the lists, for instance with a weighted score combination or reciprocal rank fusion, which has the advantage of not depending on score scale.
Filters are not a detail
In practice a query is almost never purely semantic: it is restricted to a language, a time window, a customer workspace. Filtering after the search empties the result set as soon as the filter is selective; filtering before it forces you to scan a subset the index was not built over. A usable vector engine must apply metadata predicates during graph traversal, not after. We will come back to this in a dedicated note.
Takeaway
Moving to vectors is not a replacement but a widening of recall across rephrasings, languages and paraphrase. The question to ask before migrating is not “is BM25 obsolete”, but “what share of my queries fails for lack of lexical overlap”.