Identification of minerals in the field is a task that is wrought with many challenges. Traditional approaches are prone to errors where there is no enough experience and expertise. Several existing techniques mainly make use of features of the minerals under a microscope and tend to favour a manual feature extraction pipeline. Deep learning methods can help overcome some of these hurdles and provide simple and effective ways to identify minerals. In this paper, we present an algorithm for identifying minerals from hand specimen images. Using a Convolutional Neural Network (CNN), we develop a single-label image classification model to identify and categorise seven classes of minerals. Experiments conducted using real-world datasets show that the model achieves an accuracy of 90.75%.