Adversarial examples against deep neural networks (DNNs) have been extensively developed in recent years. Modeling the distribution of adversarial perturbations could play an important role in generating adversarial perturbations, especially in the scenario of black-box adversarial attack. However, the adversarial distribution is rarely studied as far as we know. To this end, we propose to approximate the conditional distribution of adversarial perturbations given benign examples by the conditional generative flow model (c-Glow), which shows powerful ability of capturing the complex data distribution. However, the standard training of the c-Glow by maximum likelihood estimation requires massive adversarial perturbations, which is time-consuming. To address this problem, we innovatively propose to efficiently learn the c-Glow by minimizing the KL divergence between it and an energy-based model, which can evaluate the probability of being adversarial for any randomly sampled perturbation, rather than only adversarial perturbations. In this work, we propose a novel score-based black-box adversarial attack method by designing a novel transfer mechanism based on the c-Glow model pretrained with the above efficient training method on surrogate models, to take advantage of both the adversarial transferability and queries to the target model. Extensive experiments demonstrate that the proposed method is superior on both attack success rate and query efficiency to several state-of-the-art black-box attack methods.