Building change detection underpins many important applications, especially in the military and crisis management domains. Recent methods used for change detection have shifted towards deep-learning, which depends on the quality of its training data. The assembly of large-scale annotated satellite imagery datasets is therefore essential for global building change surveillance. Existing datasets almost exclusively offer near-nadir viewing angles. This limits the range of changes that can be detected. By offering larger observation ranges, the scroll imaging mode of optical satellites presents an opportunity to overcome this restriction. This paper therefore introduces S2Looking, a building change detection dataset that contains large-scale side-looking satellite images captured at various off-nadir angles. The dataset consists of 5000 bitemporal image pairs of rural areas and more than 65,920 annotated instances of changes throughout the world. The dataset can be used to train deep-learning-based change detection algorithms. It expands upon existing datasets by providing: 1) larger viewing angles; 2) large illumination variances; and 3) the added complexity of rural images. To facilitate use of the dataset, a benchmark task has been established and preliminary tests suggest deep-learning algorithms find the dataset significantly more challenging than the closest competing near-nadir dataset, LEVIR-CD+. S2Looking may therefore promote important advances in existing building change detection algorithms. The dataset is available at https://github.com/S2Looking/.