Hybrid Retrieval Explained: Combining Keyword, Vector, and Graph Search

Query your memory store for ERR_MEM_4412 and a dense retriever will hand back chunks about ERR_MEM_4413. Both strings occupy nearly the same point in embedding space. They mean entirely different things. BM25, a bag-of-words ranking function from 1994, gets this right on the first try, and no amount of embedding-model upgrade fixes the dense retriever, because the failure is structural: a 1024-dimensional float vector cannot preserve the discriminative value of a rare token that carries almost no semantic content.

Run it the other way and the roles reverse. Ask "how did we handle the memory allocation problem" against a store where the relevant entry says "fixed the heap fragmentation in the ingest worker" and BM25 returns nothing. Zero term overlap. The embedding finds it immediately.

These are not two implementations of the same idea with different accuracy. They fail on disjoint sets of queries, which is the only real justification for running both.

Fusing the two without normalizing scores

The naive approach is a weighted sum of the two scores. Do not do this. BM25 produces unbounded values that depend on corpus statistics, document length, and the number of query terms; cosine similarity lives in a narrow band, usually 0.6 to 0.9 for anything plausibly relevant. Adding them means picking a scaling constant that is valid for one corpus at one point in time and silently wrong after the next thousand inserts.

Reciprocal Rank Fusion sidesteps the problem by throwing away the scores and keeping only the ranks:

RRF(d) = sum over result lists L of  1 / (k + rank_L(d))

k = 60, rank is 1-based, documents absent from a list contribute 0

Because it consumes ranks, no normalization is needed and no retriever can dominate through scale alone. The constant k = 60 comes from the original paper and it acts as a flattener: at k = 60, rank 1 scores 1/61 = 0.0164 and rank 2 scores 1/62 = 0.0161, a 2% gap. Rank 1 in one list therefore cannot outweigh appearing at rank 3 in both lists. That is the desired behavior. Agreement between independent retrievers is stronger evidence than confidence from one.

function rrf(lists, k = 60, limit = 10) {
  const scores = new Map();
  for (const list of lists) {
    list.forEach((doc, i) => {
      const prev = scores.get(doc.id) ?? { doc, score: 0 };
      prev.score += 1 / (k + i + 1);
      scores.set(doc.id, prev);
    });
  }
  return [...scores.values()]
    .sort((a, b) => b.score - a.score)
    .slice(0, limit)
    .map((e) => e.doc);
}

That is the whole algorithm. It has no trained weights, no per-corpus tuning, and it extends to a third or fourth retriever by appending to lists. Cormack, Clarke, and Buettcher published it in 2009 and showed it beat learned fusion methods that had access to the underlying scores. It is what Elasticsearch ships for hybrid search today.

Lowering ktoward 1 sharpens the curve and lets a single list's top hit win outright; raising it flattens everything toward uniform. If you change it, change it because you measured something, not because 60 looks arbitrary.

The third retriever solves a different problem

Keyword and vector search both answer "which stored text resembles this query text." Some queries are not that question at all.

"What else broke after the pgbouncer migration" is asking for the neighbors of an entity, not for text similar to a description of that entity. The relevant memories may share no vocabulary with the query and no vocabulary with each other. What they share is an edge to the same node. A single-hop traversal from the resolved entity returns them; similarity over text does not, at any k.

You do not need a graph database for this. An edge table and a recursive CTE covers one and two hops fine in Postgres:

WITH RECURSIVE hop AS (
  SELECT target_id, 1 AS depth FROM memory_edges WHERE source_id = $1
  UNION
  SELECT e.target_id, h.depth + 1
  FROM memory_edges e JOIN hop h ON e.source_id = h.target_id
  WHERE h.depth < 2
)
SELECT m.* FROM memories m JOIN hop ON m.id = hop.target_id;

The expensive part is not traversal, it is entity resolution: mapping "the pgbouncer migration" in a user's query to a node id. That is an extra LLM call or a fuzzy match against an entity table, and it is where the latency goes.

When each one earns its cost

Every retriever you add is latency and complexity. A usable decision rule, based on the query rather than on preference:

• Vector, always. It is the only one with recall on paraphrase, which is the majority of natural-language queries. Roughly 20 to 50ms against pgvector with an HNSW index at typical memory-store sizes.
• BM25, always. A Postgres GIN index on a tsvector costs single-digit milliseconds and runs in parallel with the vector query, so the added wall-clock time is near zero. There is no scenario where the exact-identifier recovery is not worth that.
• Graph, conditionally.Only when the query contains a resolvable named entity and asks about relations ("what else," "caused by," "related to," "downstream of"). Otherwise skip it. Entity resolution plus traversal can add 100 to 300ms, and on a query with no entity to anchor it you are paying that for nothing.

The trap is running all three unconditionally because the architecture diagram looks better. On a paraphrase query the graph leg contributes noise at real latency cost, and RRF will happily blend that noise into your top 10.

Run retrievers that fail differently, fuse them on rank, and gate the expensive one on a property of the query. That is most of hybrid retrieval.

One implementation detail that saves grief: over-fetch before fusing. Pull 50 from each retriever if you intend to return 10. RRF rewards documents that appear in several lists, and a document sitting at rank 22 in one list and rank 4 in another is exactly the signal you want to catch. Truncate each list at 10 and you never see it.

Unimatrix fuses a pgvector nearest-neighbor query with Postgres full-text search on every recall, and applies the time-aware weighting after fusion rather than before. The retrieval behavior exposed through the tool surface is described in the MCP reference.