Deep generative models such as GANs and normalizing flows are powerful priors. They can regularize inverse problems to reduce ill-posedness and attain high-quality results. However, the latent vector of such deep generative models can fall out of the desired high-dimensional standard Gaussian distribution during an inversion, particularly in the presence of noise in data or inaccurate forward models. In such a case, deep generative models are ineffective in attaining high-fidelity solutions. To address this issue, we propose to reparameterize and Gaussianize the latent vector using novel differentiable data-dependent layers wherein custom operators are defined by solving optimization problems. These proposed layers constrain an inversion to find feasible in-distribution solutions. We tested and validated our technique on three inversion tasks: compressive-sensing MRI, image deblurring, and eikonal tomography (a nonlinear PDE-constrained inverse problem), using two representative deep generative models: StyleGAN2 and Glow, and achieved state-of-the-art results.