In this paper, we show that the difference in l_2 norms of sample features can hinder batch normalization from obtaining more distinguished inter-class features and more compact intra-class features. To address this issue, we propose an intuitive but effective method to equalize the l_2 norms of sample features. Concretely, we l_2-normalize each sample feature before feeding them into batch normalization, and therefore the features are of the same magnitude. Since the proposed method combines the l_2 normalization and batch normalization, we name our method L_2BN. The L_2BN can strengthen the compactness of intra-class features and enlarge the discrepancy of inter-class features. The L_2BN is easy to implement and can exert its effect without any additional parameters or hyper-parameters. Therefore, it can be used as a basic normalization method for neural networks. We evaluate the effectiveness of L_2BN through extensive experiments with various models on image classification and acoustic scene classification tasks. The results demonstrate that the L_2BN can boost the generalization ability of various neural network models and achieve considerable performance improvements.