Hybrid Retrieval and Score Fusion

Lexical retrieval rewards evidence carried by tokens and collection statistics. It is naturally strong when an exact identifier, quoted phrase, rare name, or domain term matters. Dense retrieval maps a query and passage into learned vectors, making paraphrase and semantic neighborhood available even without literal overlap. Either can fail: token mismatch can hide a relevant passage from sparse search, while an embedding can collapse distinctions, miss a newly introduced term, or retrieve a topically similar but answerless chunk.

Hybrid retrieval is useful only when the component runs add complementary relevant candidates for the target distribution. If the lexical and dense systems fail on the same queries, fusion merely rearranges shared mistakes. Begin with paired failure analysis: which judged-relevant documents appear only in one run, at what ranks, and for which query cohorts?

In the illustrative fixture, doc-a is lexical rank 1 and dense rank 2, while doc-c is lexical rank 3 and dense rank 1. With k=60, doc-a receives 1/61 + 1/62 = 0.032522 and doc-c receives 1/63 + 1/61 = 0.032266. Their unrelated raw scores never enter the calculation. A deterministic secondary key—best component rank, then document ID—makes exact ties replayable.

RRF's simplicity removes one calibration problem, not the need for evaluation. Candidate depth changes which documents are allowed to vote. The constant changes the relative influence of top ranks. Duplicated variants of one retriever can overweight one signal. Treat the set of runs and their ordering as part of the release contract.

A fused result should bind the exact query digest and the revisions of the corpus, query policy, lexical model, dense model, embedding model, evaluation cohort, and source lineage. Each hit binds document identity, document revision, content digest, rank, retriever, and raw score. If the same document ID resolves to different content in the two indexes, the artifact rejects the fusion rather than voting across an ambiguous identity.

• Require unique, contiguous ranks inside each run; duplicate documents cannot cast multiple votes.
• Apply tenant, ACL, locale, and lifecycle constraints consistently. Fusion must not broaden the caller's authority.
• Record absence honestly. Do not synthesize a worst rank for a document that a retriever never returned unless that alternative policy is explicitly evaluated.
• Hashing exposes content substitution, but upstream systems must authenticate the query and document bytes represented by those hashes.

1def fuse_rankings(contract: FusionContract, runs) -> FusionReport:
2    """Fuse rank positions only; raw lexical and dense scores never share a scale."""
3    contract = validate_record(contract, FusionContract)
4    runs = sequence(runs, 2, 2)
5    runs = tuple(validate_record(run, RetrievalRun) for run in runs)
6    if tuple(run.retriever for run in runs) != contract.retrievers:
7        raise ValueError("runs must follow the declared retriever order")

example=illustrative_only
status=FUSED_FOR_EVALUATION
order=doc-a,doc-c,doc-b
scores=0.032522,0.032266,0.016129
claim=ILLUSTRATIVE_FUSION_NOT_RETRIEVAL_BENCHMARK

Frozen records defensively copy sequences, seal all declarations, and are reconstructed at the public boundary. Tests cover stale query and scope identities, missing or reversed runs, noncontiguous ranks, duplicate documents, cross-index revision/content skew, boolean counts, NaN, infinity, subnormal values, string subclasses, direct constructor bypass, and deterministic output.

FUSED_FOR_EVALUATION authorizes only downstream evaluation of this local ordering. The script does not execute BM25, create embeddings, authenticate hashes, apply production ACLs, measure recall, or tune k. Its four invented documents cannot support a performance claim.

Evaluate sparse-only, dense-only, and fused runs on the same queries and relevance judgments. Report recall at the candidate depth that reaches the reranker, ranking metrics at the user-visible cutoff, and slices for identifiers, paraphrases, fresh content, languages, long queries, and access-filter selectivity. A global average can hide a dense regression on exact codes or a lexical regression on conversational queries.

1. 01Observe contributionRecord which retriever introduced each selected document and where both agreed; watch for one dead or dominating run.
2. 02Budget the parallel pathMeasure lexical, dense, filter, fusion, and timeout behavior separately. Define whether a missing run blocks or invokes an explicitly evaluated degraded mode.
3. 03Rollback the contractRetain the prior indexes, models, fusion parameters, and evaluation cohort together. Rolling back only k does not undo an embedding or corpus change.