Deep neural networks achieve high prediction accuracy when the train and test distributions coincide. In practice though, various types of corruptions can deviate from this setup and cause severe performance degradations. Few methods have been proposed to address generalization in the presence of unforeseen domain shifts. In this paper, we propose a misclassification-aware Gaussian smoothing approach, coupled with mixed data augmentations, for improving robustness of image classifiers against a variety of corruptions while still maintaining high clean accuracy. We show that our method improves upon the state-of-the-art in robustness and uncertainty calibration on several image classification benchmarks and network architectures.