Abstract
Deep neural networks have shown their power in the image denoising problem by learning similar patterns in natural images. However, the traditional sigmoid function has shown its limitations. In this paper, we adopt the rectified linear (ReL) function instead of the sigmoid function as the activation function of hidden layers to further enhance the ability of neural network on solving image denoising problem. Our experiment shows that by better capturing patterns in natural images, our model can achieve better performance and less time consumption than those using sigmoid units. A large number of experiments show that our approach can achieve the state-of-the-art performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR), vol. 2, pp. 60–65. IEEE (2005)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing 16(8), 2080–2095 (2007)
Aharon, M., Elad, M., Bruckstein, A.: k-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing 54(11), 4311–4322 (2006)
Ma, L., Zhang, L.: Overcomplete topographic independent component analysis. Neurocomputing 71(10), 2217–2223 (2008)
Zhao, H., Luo, J., Huang, Z., Nagumo, T., Murayama, J., Zhang, L.: Image denoising based on overcomplete topographic sparse coding. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013, Part III. LNCS, vol. 8228, pp. 266–273. Springer, Heidelberg (2013)
Burger, H., Schuler, C., Harmeling, S.: Image denoising: Can plain neural networks compete with bm3d? In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2392–2399 (June 2012)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Xie, J., Xu, L., Chen, E.: Image denoising and inpainting with deep neural networks. In: NIPS, pp. 350–358 (2012)
Jain, V., Seung, H.S.: Natural image denoising with convolutional networks. In: NIPS, vol. 8, pp. 769–776 (2008)
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 5(2), 157–166 (1994)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, JMLR W&CP, vol. 15, pp. 315–323 (2011)
Zeiler, M.D., Ranzato, M., Monga, R., Mao, M., Yang, K., Le, Q., Nguyen, P., Senior, A., Vanhoucke, V., Dean, J., et al.: On rectified linear units for speech processing. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3517–3521. IEEE (2013)
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of the ICML (2013)
Lecun, Y., Bottou, L., Orr, G.B., Müller, K.R.: Efficient backprop (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Wu, Y., Zhao, H., Zhang, L. (2014). Image Denoising with Rectified Linear Units. In: Loo, C.K., Yap, K.S., Wong, K.W., Beng Jin, A.T., Huang, K. (eds) Neural Information Processing. ICONIP 2014. Lecture Notes in Computer Science, vol 8836. Springer, Cham. https://doi.org/10.1007/978-3-319-12643-2_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-12643-2_18
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12642-5
Online ISBN: 978-3-319-12643-2
eBook Packages: Computer ScienceComputer Science (R0)