Visual Simultaneous Localization and Mapping (SLAM) has received significant attention in recent years due to its ability to estimate camera trajectory and create an environment map using visual data alone, making a substantial contribution to autonomous driving applications, in particular, a real-world scenario with moving crowds and vehicles. In this work, we propose a visual SLAM system that incorporates moving object trajectory tracking and prediction. We take into account the motion clues of the pedestrians to track and predict their movement, as long as mapping the environment. Such an integrated system solves the localization of the camera and other moving objects in the scene, and further creates a sparse map to support the potential navigation of the vehicle. In the experiment, we demonstrate the effectiveness and robustness of our approach through a comprehensive evaluation on both our simulation and real-world KITTI datasets.