The low-rank tensor approximation is very promising for the compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our approach improves the compression rate while maintaining the accuracy for a variety of tasks.
updated: Fri Nov 15 2019 12:27:25 GMT+0000 (UTC)
published: Sun Mar 24 2019 11:40:18 GMT+0000 (UTC)