LAMAR: An Open Language-Aware Multilingual Alignment Reranker

When building Retrieval-Augmented Generation (RAG) systems that operate across multiple languages, we often assume that finding a semantically relevant document is enough. However, the reality is messier. A retriever might pull a perfectly relevant document in Chinese for a Spanish query. While the meaning is there, the downstream LLM might struggle to synthesize an accurate answer from a language mismatch.

Current multilingual rerankers aim to solve the semantic relevance problem—ensuring the document matches the query's intent. But they often ignore "language coherence" (the principle that the document should match the query's language). This paper introduces LAMAR, a reranker designed to prioritize same-language documents when semantically equivalent candidates are available.

The failure of language-agnostic relevance

Most multilingual rerankers focus on a single goal: semantic relevance. In this paradigm, a German document and an English document are treated as equally valid if they both satisfy the query. The authors argue this is a mistake for RAG pipelines. As shown in, documents written in the same language as the query yield higher F1 scores (a measure of accuracy) during answer generation.

The problem is that existing rerankers do not inherently respect this linguistic preference. The authors conducted a diagnostic evaluation using the XQuAD dataset. They created "oracle" sets (controlled groups where every candidate is semantically equivalent) to test this. The results in are telling.

Even high-performing models like Jina-reranker-v3 fail to prioritize the query language for English queries. Instead, they often rank Chinese or Thai documents at the top. This lack of coherence means your reranker might push the most useful documents down the context window.

Two stages of linguistic alignment

To fix this, the authors propose LAMAR, a cross-encoder (a model that processes query and document pairs simultaneously) trained via a two-stage process.

1. English-anchored relevance distillation: The first step ensures relevance scores are consistent across languages. The authors use an English "anchor" to calibrate the student model. For every multilingual pair, the student is trained to match the score assigned by a teacher model to the corresponding English pair. This forces the model to map multilingual semantics into a unified scoring space.

2. Preference alignment for language coherence: Once the model understands relevance, the second stage injects language preference. The authors use a joint objective combining a group ranking loss and a language-coherence loss ($L_{LLC}$). They construct parallel document groups containing both positive and negative documents in both the source and target languages. The $L_{LLC}$ component uses a softplus function (a smooth mathematical function) to penalize the model if it fails to rank same-language documents higher. Crucially, the group ranking loss ensures the model still respects the hierarchy of relevance (positives > negatives).

High coherence, low parameter cost

The performance delta is most visible in the controlled language-coherence experiments. The authors report that LAMAR achieves the best performance across all examined languages on the XQuAD and BELEBELE benchmarks. It reaches an nDCG@1 (a metric measuring if the top result is correct) of 0.9689 on XQuAD and 0.9466 on BELEBELE [Table 3]. Looking at, LAMAR maintains higher and more stable scores across different query languages.

It avoids the volatility seen in heavier models like Llama-3.3-70B or Qwen3-4B.

What makes this compelling for production is the efficiency. LAMAR is a 0.6B parameter model. Despite its small footprint, it performs comparably to much larger rerankers on standard multilingual benchmarks like MIRACL and MTEB [Table 4, Table 5]. In practical "top-20" retrieval scenarios, LAMAR hits the highest marks across all reported metrics [Table 6]. It provides high-level multilingual intelligence at a fraction of the inference cost and latency.

Assessing the blind spots

While the results are strong, there are nuances to consider. First, the language-coherence advantage is most clearly demonstrated in "oracle" settings. In these tests, all candidates are semantically equivalent. In a messy production environment, your retriever might return a high-relevance document in the target language and a completely irrelevant one in the query language. It is not immediately clear how LAMAR's preference alignment handles such extreme semantic noise.

Second, the training relies heavily on parallel datasets like MIRACL. While the authors use data synthesis to expand coverage, the model's ability to generalize to extremely low-resource languages remains unproven. If your RAG application targets a niche dialect, the "coherence" might degrade back to simple semantic relevance.

The verdict

If you are running multilingual RAG, LAMAR is a strong candidate. The core insight—that rerankers should treat language consistency as a priority alongside semantic relevance—is empirically validated. A 0.6B model that outperforms 4B models while providing better language stability is a massive win for managing GPU budgets.

The model is available at https://huggingface.co/nlpai-lab/LAMAR-600m. If you see poor reasoning in your multilingual outputs, the culprit might be your reranker. It may be feeding the LLM the right information in the wrong language. Try swapping in LAMAR as a quick prototype.

Figures from the paper