Cervical abnormal cell detection is a challenging task as the morphological differences between abnormal cells and normal cells are usually subtle. To determine whether a cervical cell is normal or abnormal, cytopathologists always take surrounding cells as references and make careful comparison to identify its abnormality. To mimic these clinical behaviors, we propose to explore contextual relationships to boost the performance of cervical abnormal cell detection. Specifically, both contextual relationships between cells and cell-to-global images are exploited to enhance features of each region of interest (RoI) proposals. Accordingly, two modules, termed as RoI-relationship attention module (RRAM) and global RoI attention module (GRAM) are developed and their combination strategies are also investigated. We setup strong baselines by using single-head or double-head Faster R-CNN with feature pyramid network (FPN) and integrate our RRAM and GRAM into them to validate the effectiveness of the proposed modules. Experiments conducted on a large cervical cell detection dataset consisting of 40,000 cytology images reveal that the introduction of RRAM and GRAM both achieves better average precision (AP) than the baseline methods. Moreover, when cascading RRAM and GRAM, our method outperforms the state-of-the-art (SOTA) methods. Furthermore, we also show the proposed feature enhancing scheme can facilitate the image-level and smear-level classification. The code and trained models are publicly available at https://github.com/CVIU-CSU/CR4CACD.