Convolutional neural networks (CNNs) have developed to become powerful models for various computer vision tasks ranging from object detection to semantic segmentation. However, most of state-of-the-art CNNs can not be deployed directly on edge devices such as smartphones and drones, which need low latency under limited power and memory bandwidth. One popular, straightforward approach to compressing CNNs is network slimming, which imposes ℓ_1 regularization on the channel-associated scaling factors via the batch normalization layers during training. Network slimming thereby identifies insignificant channels that can be pruned for inference. In this paper, we propose replacing the ℓ_1 penalty with an alternative sparse, nonconvex penalty in order to yield a more compressed and/or accurate CNN architecture. We investigate ℓ_p (0 < p < 1), transformed ℓ_1 (Tℓ_1), minimax concave penalty (MCP), and smoothly clipped absolute deviation (SCAD) due to their recent successes and popularity in solving sparse optimization problems, such as compressed sensing and variable selection. We demonstrate the effectiveness of network slimming with nonconvex penalties on VGGNet, Densenet, and Resnet on standard image classification datasets. Based on the numerical experiments, Tℓ_1 preserves model accuracy against channel pruning, ℓ_1/2, 3/4 yield better compressed models with similar accuracies after retraining as ℓ_1, and MCP and SCAD provide more accurate models after retraining with similar compression as ℓ_1. Network slimming with Tℓ_1 regularization also outperforms the latest Bayesian modification of network slimming in compressing a CNN architecture in terms of memory storage while preserving its model accuracy after channel pruning.