Generative Adversarial Networks (GANs) are the most popular image generation models that have achieved remarkable performance on various tasks. However, training instability is still one of the open problems for all GAN-based algorithms. In order to stabilize GANs training, some regularization and normalization techniques have been proposed to make the discriminator meet the Lipschitz continuity. In this paper, a new approach inspired by works on adversarial is proposed to stabilize the training process of GANs. It is found that sometimes the images generated by the generator play a role just like adversarial examples for discriminator during the training process, which might be a part of the reason for the unstable training of GANs. With this discovery, we propose a Direct Adversarial Training (DAT) method for the training process of GANs to improve its performance. We prove that the DAT method can minimize the Lipschitz constant of the discriminator adaptively. The advanced performance of the proposed method is verified on multiple baseline and SOTA networks, such as DCGAN, Spectral Normalization GAN, Self-supervised GAN, and Information Maximum GAN. Code will be available at https://github.com/iceli1007/DAT-GAN