Class imbalance is a challenging issue in practical classification problems for deep learning models as well as for traditional models. Traditionally successful countermeasures such as synthetic over-sampling have had limited success with complex, structured data handled by deep learning models. In this work, we propose to use a Generative Adversarial Network (GAN) equipped with a generator network G, a discriminator network D and a classifier network C to remove the class-imbalance in visual data sets. The generator network is initialized with auto-encoder to make it stable. The discriminator D ensures that G adheres to class distribution of imbalanced class. In conventional methods, where Generator G competes with discriminator D in a min-max game, we propose to further add an additional classifier network to the original network. Now, the generator network tries to compete in a min-max game with Discriminator as well as the new classifier that we have introduced. An additional condition is enforced on generator network G to produce points in the convex hull of desired imbalanced class. Further the contention of adversarial game with classifier C, pushes conditional distribution learned by G towards the periphery of the respective class, compensating the problem of class imbalance. Experimental evidence shows that this initialization results in stable training of the network. We achieve state of the art performance on extreme visual classification task on the FashionMNIST, MNIST, SVHN, ExDark, MVTec Anomaly Detection dataset, Chest X-Ray dataset and others.