Classification models trained on biased datasets usually perform poorly on out-of-distribution samples since biased representations are embedded into the model. Recently, various debiasing methods have been proposed to disentangle biased representations, but it is challenging to discard only the biased features without altering other relevant information. In this paper, we propose a novel debiasing approach that explicitly generates additional images using texture representations of different labeled images to enlarge the training dataset and mitigate the bias effects when training a classifier. Every new generated image contains similar content information from a source image while transferring textures from a target image with a different label. Our model includes a texture co-occurrence loss that determines whether a generated image's texture is similar to that of the target, and a spatial self-similarity loss that determines whether the content details between the generated and source images are well preserved. Both generated and original training images are further used to train a classifier that is able to improve robustness against inconsistent bias representations. We employ five distinct artificially designed datasets with known biases to demonstrate the ability of our method to mitigate bias information. For all cases, our method outperformed existing state-of-the-art methods. Codes are available at: https://github.com/myeongkyunkang/i2i4debias