Anomaly detection in video streams is a challengingproblem because of the scarcity of abnormal events andthe difficulty of accurately annotating them.To allevi-ate these issues, unsupervised learning-based predictionmethods have been previously applied. These approachestrain the model with only normal events and predict a fu-ture frame from a sequence of preceding frames by use ofencoder-decoder architectures so that they result in smallprediction errors on normal events but large errors on ab-normal events. The architecture, however, comes with thecomputational burden as some anomaly detection tasks re-quire low computational cost without sacrificing perfor-mance. In this paper, Cross-Parallel Network (CPNet) forefficient anomaly detection is proposed here to minimizecomputations without performance drops. It consists ofNsmaller parallel U-Net, each of which is designed to handlea single input frame, to make the calculations significantlymore efficient. Additionally, an inter-network shift moduleis incorporated to capture temporal relationships among se-quential frames to enable more accurate future predictions.The quantitative results show that our model requires lesscomputational cost than the baseline U-Net while deliver-ing equivalent performance in anomaly detection.