Digital Elevation Models (DEMs) are indispensable in the fields of remote sensing and photogrammetry, with their refinement and enhancement being critical for a diverse array of applications. Numerous methods have been developed for enhancing DEMs, but most of them concentrate on tackling specific tasks individually. This paper presents a unified generative model for multipurpose DEM restoration, diverging from the conventional approach that typically targets isolated tasks. We modify the mean-reverting stochastic differential equation, to generally refine the DEMs by conditioning on the learned terrain priors. The proposed Efficient Terrain Stochastic Differential Equation (ET-SDE) models DEM degradation through SDE progression and restores it via a simulated reversal process. Leveraging efficient submodules with lightweight channel attention, this adapted SDE boosts DEM quality and streamlines the training process. The experiments show that ET-SDE achieves highly competitive restoration performance on super-resolution, void filling, denoising, and their combinations, compared to the state-of-the-art work. In addition to its restoration capabilities, ET-SDE also demonstrates faster inference speeds and the capacity to generalize across various tasks, particularly for larger patches of DEMs.