Identifying an appropriate underlying graph kernel that reflects pairwise similarities is critical in many recent graph spectral signal restoration schemes, including image denoising, dequantization, and contrast enhancement. Existing graph learning algorithms compute the most likely entries of a properly defined graph Laplacian matrix L, but require a large number of signal observations z's for a stable estimate. In this work, we assume instead the availability of a relevant feature vector f_i per node i, from which we compute an optimal feature graph via optimization of a feature metric. Specifically, we alternately optimize the diagonal and off-diagonal entries of a Mahalanobis distance matrix M by minimizing the graph Laplacian regularizer (GLR) z^ L z, where edge weight is w_{i,j = -(\mathbf{f_i - f_j)^ M (f_i - f_j) \, given a single observation z. We optimize diagonal entries via proximal gradient (PG), where we constrain M to be positive definite (PD) via linear inequalities derived from the Gershgorin circle theorem. To optimize off-diagonal entries, we design a block descent algorithm that iteratively optimizes one row and column of M. To keep M PD, we constrain the Schur complement of sub-matrix M_2,2 of M to be PD when optimizing via PG. Our algorithm mitigates full eigen-decomposition of M, thus ensuring fast computation speed even when feature vector f_i has high dimension. To validate its usefulness, we apply our feature graph learning algorithm to the problem of 3D point cloud denoising, resulting in state-of-the-art performance compared to competing schemes in extensive experiments.