Modern CNN-based face detectors have achieved tremendous strides due to large annotated datasets. However, misaligned results with high detection confidence but low localization accuracy restrict the further improvement of detection performance. In this paper, we first generate detection results on training set itself. Surprisingly, a considerable part of them exist the same misalignment problem. Then, we carefully examine these misaligned cases and point out annotation inconsistency is the main reason. Finally, we propose a novel Bounding-Box Deep Calibration (BDC) method to reasonably replace inconsistent annotations with model predicted bounding-boxes and create a new annotation file for training set. Extensive experiments on WIDER FACE dataset show the effectiveness of BDC on improving models' precision and recall rate. Our simple and effective method provides a new direction for improving face detection. Source code is available at https://github.com/shiluo1990/BDC.