Beam Search for Machine Translation

How Greedy, Exhaustive, and Beam Search Algorithms Work

Language Models

October 17, 2021

Beam Search for Machine Translation: How Greedy, Exhaustive, and Beam Search Algorithms Work

Beam search is a heuristic technique for choosing the most likely output used by probabilistic sequence models like neural machine translation.

Machine translation is fundamentally multi-label learning where we have multiple plausible answers, yet the model must pick the best possible translation.

We often use greedy search while explaining machine translation models, focusing on core concepts such as the RNN encoder-decoder structure and attention mechanisms. So, we do not talk much about actual output generation.

However, it is a common assumption that we should already know what beam search is. As such, people new to the field might get stuck with the so-called minor detail before reaching the main subject in a paper.

This article discusses how beam search works with the following topics:

4 References