Mitotic count is the most important morphological feature of breast cancer grading. Many deep learning-based methods have been proposed but suffer from domain shift. In this work, we construct a Fourier-based segmentation model for mitosis detection to address the problem. Swapping the low-frequency spectrum of source and target images is shown effective to alleviate the discrepancy between different scanners. Our Fourier-based segmentation method can achieve F1 with 0.7456 on the preliminary test set.