Agricultural image recognition tasks are becoming increasingly dependent on deep learning (DL); however, despite the excellent performance of DL, it is difficult to comprehend the type of logic or features of the input image it uses during decision making. Knowing the logic or features is highly crucial for result verification, algorithm improvement, training data improvement, and knowledge extraction. However, the explanations from the current heatmap-based algorithms are insufficient for the abovementioned requirements. To address this, this paper details the development of a classification and explanation method based on a variational autoencoder (VAE) architecture, which can visualize the variations of the most important features by visualizing the generated images that correspond to the variations of those features. Using the PlantVillage dataset, an acceptable level of explainability was achieved without sacrificing the classification accuracy. The proposed method can also be extended to other crops as well as other image classification tasks. Further, application systems using this method for disease identification tasks, such as the identification of potato blackleg disease, potato virus Y, and other image classification tasks, are currently being developed.