Convolutional neural network (CNN) has surpassed traditional methods for med-ical image classification. However, CNN is vulnerable to adversarial attacks which may lead to disastrous consequences in medical applications. Although adversarial noises are usually generated by attack algorithms, white-noise-induced adversarial samples can exist, and therefore the threats are real. In this study, we propose a novel training method, named IMA, to improve the robust-ness of CNN against adversarial noises. During training, the IMA method in-creases the margins of training samples in the input space, i.e., moving CNN de-cision boundaries far away from the training samples to improve robustness. The IMA method is evaluated on four publicly available datasets under strong 100-PGD white-box adversarial attacks, and the results show that the proposed meth-od significantly improved CNN classification accuracy on noisy data while keep-ing a relatively high accuracy on clean data. We hope our approach may facilitate the development of robust applications in medical field.