This paper proposes an unsupervised anomaly detection technique for image-based plant disease diagnosis. A construction of large and openly available data set on labeled images of healthy and diseased crop plants has led to growing interest in computer vision techniques for plant disease diagnosis. Although supervised image classifiers based on deep learning could be a powerful tool to identify plant diseases, they require huge amount of data set that have been labeled as healthy and disease. While, data mining techniques called anomaly detection includes unsupervised approaches that not require rare samples for training classifiers The proposed method in this study focuses on the reconstructability of colors on plant images. We expect that a deep encoder decoder network trained for reconstructing colors of healthy plant images fails to color symptomatic regions. The main contributions of this work are as follows: (i) we propose a new image-based plant disease detection framework utilising a conditional adversarial network called pix2pix,(ii) we introduce a new anomaly score calculated from CIEDE2000 color difference. Through experiments using the PlantVillage dataset, we demonstrate that our method is superior to an existing anomaly detector called AnoGAN for identifying diseased crop images in terms of accuracy, interpretability and computational efficiency.