Diffusion models are a class of deep generative models that have shown impressive results on various tasks with dense theoretical founding. Although diffusion models have achieved more impressive quality and diversity of sample synthesis than other state-of-the-art models, they still suffer from costly sampling procedures and sub-optimal likelihood estimation. Recent studies have shown great enthusiasm for improving the performance of the diffusion model. In this article, we present the first comprehensive review of existing variants of diffusion models. Specifically, we provide the first taxonomy of diffusion models and categorize them into three types: sampling-acceleration enhancement, likelihood-maximization enhancement, and data-generalization enhancement. We also introduce the other five generative models (i.e., variational autoencoders, generative adversarial networks, normalizing flow, autoregressive models, and energy-based models) in detail and clarify the connections between diffusion models and these generative models. Then we thoroughly investigate the applications of diffusion models, including computer vision, natural language processing, waveform signal processing, multi-modal modeling, molecular graph generation, time series modeling, and adversarial purification. Furthermore, we propose new perspectives pertaining to the development of this generative model.