Automatic recognition and segmentation methods now become the essential requirement in identifying co-seismic landslides, which are fundamental for disaster assessment and mitigation in large-scale earthquakes. This approach used to be carried out through pixel-based or object-oriented methods. However, due to the massive amount of remote sensing data, variations in different earthquake scenarios, and the efficiency requirement for post-earthquake rescue, these methods are difficult to develop into an accurate, rapid, comprehensive, and general (cross-scene) solution for co-seismic landslide recognition. This paper develops a robust model, Dense Feature Pyramid with Encoder-decoder Network (DFPENet), to understand and fuse the multi-scale features of objects in remote sensing images. The proposed method achieves a competitive segmentation accuracy on the public ISPRS 2D Semantic. Furthermore, a comprehensive and widely-used scheme is proposed for co-seismic landslide recognition, which integrates image features extracted from the DFPENet model, geologic features, temporal resolution, landslide spatial analysis, and transfer learning, while only RGB images are used. To corroborate its feasibility and applicability, the proposed scheme is applied to two earthquake-triggered landslides in Jiuzhaigou (China) and Hokkaido (Japan), using available pre- and post-earthquake remote sensing images.