Tensor completion and robust principal component analysis have been widely used in machine learning while the key problem relies on the minimization of a tensor rank that is very challenging. A common way to tackle this difficulty is to approximate the tensor rank with the ℓ_1-norm of singular values based on its Tensor Singular Value Decomposition (T-SVD). Besides, the sparsity of a tensor is also measured by its ℓ_1-norm. However, the ℓ_1 penalty is essentially biased and thus the result will deviate. In order to sidestep the bias, we propose a novel non-convex tensor rank surrogate function and a novel non-convex sparsity measure. In this new setting by using the concavity instead of the convexity, a majorization minimization algorithm is further designed for tensor completion and robust principal component analysis. Furthermore, we analyze its theoretical properties. Finally, the experiments on natural and hyperspectral images demonstrate the efficacy and efficiency of our proposed method.