Generative adversarial networks (GANs)successfully generate high quality data by learning amapping from a latent vector to the data. Various studies assert that the latent space of a GAN is semanticallymeaningful and can be utilized for advanced data analysis and manipulation. To analyze the real data in thelatent space of a GAN, it is necessary to build an inference mapping from the data to the latent vector. Thispaper proposes an effective algorithm to accurately infer the latent vector by utilizing GAN discriminator features. Our primary goal is to increase inference mappingaccuracy with minimal training overhead. Furthermore,using the proposed algorithm, we suggest a conditionalimage generation algorithm, namely a spatially conditioned GAN. Extensive evaluations confirmed that theproposed inference algorithm achieved more semantically accurate inference mapping than existing methodsand can be successfully applied to advanced conditionalimage generation tasks.