Recently deep neural networks based on tanh activation function have shown their impressive power in image denoising. In this letter, we try to use rectifier function instead of tanh and propose a dual-pathway rectifier neural network by combining two rectifier neurons with reversed input and output weights in the same hidden layer. We drive the equivalent activation function and compare it to some typical activation functions for image denoising under the same network architecture. The experimental results show that our model achieves superior performances faster especially when the noise is small.