Current VHR(Very High Resolution) satellite images enable the detailed monitoring of the earth and can capture the ongoing works of railway construction. In this paper, we present an integrated framework applied to monitoring the railway construction in China, using QuickBird, GF-2 and Google Earth VHR satellite images. We also construct a novel DCNNs-based (Deep Convolutional Neural Networks) semantic segmentation network to label the temporary works such as borrow & spoil area, camp, beam yard and ESAs(Environmental Sensitive Areas) such as resident houses throughout the whole railway construction project using VHR satellite images. In addition, we employ HED edge detection sub-network to refine the boundary details and attention cross entropy loss function to fit the sample class disequilibrium problem. Our semantic segmentation network is trained on 572 VHR true color images, and tested on the 15 QuickBird true color images along Ruichang-Jiujiang railway during 2015-2017. The experiment results show that compared with the existing state-of-the-art approach, our approach has obvious improvements with an overall accuracy of more than 80%.