Despite unconditional feature inverters being the foundation of many synthesis tasks, training them requires a large computational overhead, decoding capacity or additional autoregressive priors. We propose to train an adversarially robust encoder to learn disentangled and perceptually-aligned bottleneck features, making them easily invertible. Then, by training a simple generator with the mirror architecture of the encoder, we achieve superior reconstructions and generalization over standard approaches. We exploit such properties using an encoding-decoding network based on AR features and demonstrate its oustanding performance on three applications: anomaly detection, style transfer and image denoising. Comparisons against alternative learn-based methods show that our model attains improved performance with significantly less training parameters.