This work proposes a method for depth completion of sparse LiDAR data using a convolutional neural network which can be used to generate semi-dense depth maps and "almost" full 3D point-clouds with significantly lower root mean squared error (RMSE) over state-of-the-art methods. We add an "Error Prediction" unit to our network and present a novel and simple end-to-end method that learns to predict an error-map of depth regression task. An "almost" dense high-confidence/low-variance point-cloud is more valuable for safety-critical applications specifically real-world autonomous driving than a full point-cloud with high error rate and high error variance. Using our predicted error-map, we demonstrate that by up-filling a LiDAR point cloud from 18,000 points to 285,000 points, versus 300,000 points for full depth, we can reduce the RMSE error from 1004 to 399. This error is approximately 60% less than the state-of-the-art and 50% less than the state-of-the-art with RGB guidance (we did not use RGB guidance in our algorithm). In addition to analyzing our results on Kitti depth completion dataset, we also demonstrate the ability of our proposed method to extend to new tasks by deploying our "Error Prediction" unit to improve upon the state-of-the-art for monocular depth estimation. Codes and demo videos are available at http://github.com/hekmak/Conf-net.