Contrastive learning is effective at learning useful representations without supervision. Yet contrastive learning is susceptible to shortcuts -- i.e., it may learn shortcut features irrelevant to the downstream task and discard relevant information. Past work has addressed this limitation via handcrafted data augmentations that eliminate the shortcut. However, handcrafted augmentations are infeasible for data modalities that are not interpretable by humans (e.g., radio signals). Further, even when the modality is interpretable (e.g., RGB), sometimes eliminating the shortcut information may be undesirable. For example, in multi-attribute classification, information related to one attribute may act as a shortcut around other attributes. This paper presents reconstructive contrastive learning (RCL), a framework for learning unsupervised representations that are robust to shortcuts. The key idea is to force the learned representation to reconstruct the input, which naturally counters potential shortcuts. Extensive experiments verify that RCL is highly robust to shortcuts and outperforms state-of-the-art contrastive learning methods on both RGB and RF datasets for a variety of tasks.