We propose to directly compute classification estimates by learning features encoded with their class scores. Our resulting model has a encoder-decoder structure suitable for supervised learning, it is computationally efficient and performs well for classification on several datasets.