Video super-resolution reconstruction (SRR) algorithms attempt to reconstruct high-resolution (HR) video sequences from low-resolution observations. Although recent progress in video SRR has significantly improved the quality of the reconstructed HR sequences, it remains challenging to design SRR algorithms that achieve good quality and robustness at a small computational complexity, being thus suitable for online applications. In this paper, we propose a new adaptive video SRR algorithm that achieves state-of-the-art performance at a very small computational cost. Using a nonlinear cost function constructed considering characteristics of typical innovation outliers in natural image sequences and an edge-preserving regularization strategy, we achieve state-of-the-art reconstructed image quality and robustness. This cost function is optimized using a specific alternating projections strategy over non-convex sets that is able to converge in a very few iterations. An accurate and very efficient approximation for the projection operations is also obtained using tools from multidimensional multirate signal processing. This solves the slow convergence issue of stochastic gradient-based methods while keeping a small computational complexity. Simulation results with both synthetic and real image sequences show that the performance of the proposed algorithm is similar or better than state-of-the-art SRR algorithms, while requiring only a small fraction of their computational cost.