Future Frame Synthesis (FFS) aims to enable models to generate sequences of future frames based on existing content. This task has garnered widespread application across various domains. In this paper, we comprehensively survey both historical and contemporary works in this field, encompassing widely used datasets and algorithms. Our survey scrutinizes the challenges and the evolving landscape of FFS within the realm of computer vision. We propose a novel taxonomy centered on the stochastic nature of related algorithms. This taxonomy emphasizes the gradual transition from deterministic to generative synthesis methodologies, highlighting significant advancements and shifts in approach.