Decoding and Text Generation

Beam search is a heuristic algorithm used to perform graph-search. Stochastic Beam Search was introduced as a heuristic to try and obtain a Maximum-A-Posteriori (MAP) solution to the problem of generating sequences from an LLM. The key problem with Beam Search is that it tends to produce outputs with repetitions and to get stuck in loops. Most importantly, there is a lack of diversity which can be attributed to the greedy optimization.