It has been an important problem to design a proper discriminator for conditional generative adversarial networks (cGANs). In this paper, we investigate two popular choices, the projection-based and classification-based discriminators, and reveal that both of them suffer some kind of drawbacks that affect the learning ability of cGANs. Then, we present our solution that trains a powerful discriminator and avoids over-fitting with regularization. In addition, we unify multiple targets (class, domain, reality, etc.) into one loss function to enable a wider range of applications. Our algorithm, named Omni-GAN, by proposing a simple modification, improves the projection-based cGAN performance significantly and achieves a new state-of-the-art in generating mid/high-resolution images (a record-breaking IS of 190.9 on ImageNet 128×128). More importantly, we explain experimentally why Omni-GAN is significantly better than the projection-based cGAN, BigGAN, offering new possible directions for optimizing cGANs. Code is available at https://github.com/PeterouZh/Omni-GAN-PyTorch.