Multi-view classification is inspired by the behavior of humans, especially when fine-grained features or in our case rarely occurring anomalies are to be detected. Current contributions point to the problem of how high-dimensional data can be fused. In this work, we build upon the deep support vector data description algorithm and address multi-perspective anomaly detection using three different fusion techniques i.e. early fusion, late fusion, and late fusion with multiple decoders. We employ different augmentation techniques with a denoising process to deal with scarce one-class data, which further improves the performance (ROC AUC = 80%). Furthermore, we introduce the dices dataset that consists of over 2000 grayscale images of falling dices from multiple perspectives, with 5% of the images containing rare anomalies (e.g. drill holes, sawing, or scratches). We evaluate our approach on the new dices dataset using images from two different perspectives and also benchmark on the standard MNIST dataset. Extensive experiments demonstrate that our proposed approach exceeds the state-of-the-art on both the MNIST and dices datasets. To the best of our knowledge, this is the first work that focuses on addressing multi-perspective anomaly detection in images by jointly using different perspectives together with one single objective function for anomaly detection.