Advertisement

Reconstruction of missing color-channel data using a three-step back propagation neural network

  • Jin Wang
  • Marco Anisetti
  • Gwanggil Jeon
Original Article

Abstract

Demosaicking aims to approximate missing color pixels through analysis of the geometric structure between given color pixels and missing color pixels. In this paper, we introduce an efficient adaptive demosaicking method based on back propagation (BP) neural network (BP-NN). We firstly reconstruct the green channel using one BPNN, and then refine the green channel utilizing another BPNN based on the color difference. With the whole green channel interpolated, we reconstruct the red/blue channel using the color difference between the green channel and red/blue channel in a local region. Finally, we refine the red/blue channel using the third BPNN. Regarding the interpolation issue, different image features have completely different properties, such as smooth regions, edges, and textures. Consequently, it is necessary to identify an adaptive model to estimate the relation among neighboring color pixels. We provide the adaptive BP-NN based demosaicking algorithm which can reduce blurring through recovery of missing pixels by a learning process, and also use a pre-trained fixed network to reduce computational complexity. Experimental results demonstrate that the proposed method outperforms extant approaches in PSNR, computational complexity, and visual quality.

Keywords

BP neural network Demosaicking Color channel Real time process 

Notes

Acknowledgment

This paper is sponsored by National Natural Science Foundation of China (No. 61501359, 61771378), by the Framework of International Cooperation Program managed by the NRF of Korea under Grant NRF-2016K1A3A1A25003543 and by the “Ministero degli Affari Esteri e della Cooperazione Internazionale” of Italy under Grant PGR00217.

References

  1. 1.
    Lukac R, Plataniotis KN, Hatzinakos D (2005) Color image zooming on the Bayer pattern. IEEE Trans Circuits Syst Video Technol 15(11):1475–1492CrossRefGoogle Scholar
  2. 2.
    Gunturk BK, Glotzbach J, Altunbask Y, Schafer RW, Mersereau RM (2005) Demosaicing: color filter array interpolation. IEEE Signal Process Mag 22(1):44–54CrossRefGoogle Scholar
  3. 3.
    Hamilton JF, Adams JE (1997) Adaptive color plane interpolation in single sensor color electronic camera. US Patent 5 629 734Google Scholar
  4. 4.
    Pei S-C, Tam I-K (2003) Effective color interpolation in CCD color filter arrays using signal correlation. IEEE Trans Circuits Syst Video Technol 13(6):503–513CrossRefGoogle Scholar
  5. 5.
    Chang L, Tam Y-P (2004) Effective use of spatial and spectral correlations for color filter array demosaicing. IEEE Trans Consum Electron 50(1):355–365CrossRefGoogle Scholar
  6. 6.
    Hirakawa K, Parks TW (2005) Adaptive homogeneity-directed demosaicing algorithm. IEEE Trans Image Process 14(3):360–369CrossRefGoogle Scholar
  7. 7.
    Lian NX, Chang L, Tan YP, Zagorodnov V (2007) Adaptive filtering for color filter array demosaicking. IEEE Trans Image Process 16(10):2515–2525MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chung K-H, Chan Y-H (2006) Color demosaicing using variance of color differences. IEEE Trans Image Process 15(10):2944–2955CrossRefGoogle Scholar
  9. 9.
    Menon D, Andriani S, Calvagno G (2007) Demosaicing with directional filtering and a posteriori decision. IEEE Trans Image Process 16(1):132–141MathSciNetCrossRefGoogle Scholar
  10. 10.
    Dengwen Z, Xiaoliu X, Weiming D (2012) Colour demosaicking with directional filtering and weighting. IET Image Process 6(8):1084–1092MathSciNetCrossRefGoogle Scholar
  11. 11.
    Li JSJ, Randhawa S (2009) Color filter array demosaicking using high-order interpolation techniques with a weighted median filter for sharp color edge preservation. IEEE Trans Image Process 18(9):1946–1957MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Su C-Y, Kao W-C (2009) Effective demosaicing using subband correlation. IEEE Trans Consum Electron 55(1):199–204CrossRefGoogle Scholar
  13. 13.
    Chen W-J, Chang P-Y (2012) Effective demosaicking algorithm based on edge property for color filter arrays. Digit Signal Process 22(1):163–169MathSciNetCrossRefGoogle Scholar
  14. 14.
    Pekkucuksen I, Altunbasak Y (2012) Edge strength filter-based color filter array interpolation. IEEE Trans Image Process 21(1):393–397MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Pekkucuksen I, Altunbasak Y (2013) Multiscale gradients-based color filter array interpolation. IEEE Trans Image Process 22(1):157–165MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Kim J, Jeon G, Jeong J (2014) Demosaicking using geometric duality and dilated directional differentiation. Optics Commun 324:194–201CrossRefGoogle Scholar
  17. 17.
    Menon D, Calvagno G (2009) Regularization approaches to demosaicking. IEEE Trans Image Process 18(10):2209–2220MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Dubois E (2005) Frequency-domain methods for demosaicking of Bayer-sampled color images. IEEE Signal Process Lett 12(12):847–850CrossRefGoogle Scholar
  19. 19.
    Leung B, Jeon G, Dubois E (2011) Least-squares luma-chroma demultiplexing algorithm for Bayer demosaicking. IEEE Trans Image Process 20(7):1885–1894MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Jeon G, Dubois E (2013) Demosaicking of noisy Bayer-sampled color images with least-squares luma-chroma demultiplexing and noise level estimation. IEEE Trans Image Process 22(1):146–156MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Chen X, Jeon G, Jeong J, He L (2015) Multidirectional weighted interpolation and refinement method for Bayer pattern CFA demosaicking. IEEE Trans Circuits Syst Video Technol 25(8):1271–1282CrossRefGoogle Scholar
  22. 22.
    Wang L, Jeon G (2015) Bayer pattern CFA demosaicking based on multi-directional weighted interpolation and guided filter. IEEE Signal Process Lett 22(11):2083–2087CrossRefGoogle Scholar
  23. 23.
    Wu J, Anisetti M, Wu W, Damiani E, Jeon G (2016) Bayer demosaicking with polynomial interpolation. IEEE Trans Image Process 25(11):5369–5382MathSciNetCrossRefGoogle Scholar
  24. 24.
    Wang J, Wu J, Wu Z, Jeon G, Jeong J (2017) Bilateral filtering and directional differentiation for Bayer demosaicking. IEEE Sens J 17(3):726–734CrossRefGoogle Scholar
  25. 25.
    Wang J, Wu J, Wu Z, Jeon G (2016) Taylor series and adaptive directional selection for real time demosaicking. Displays 45:14–25CrossRefGoogle Scholar
  26. 26.
    Seo G, Choi H, Lee C (2009) Efficient implementation of neural network deinterlacing. In: Proc. SPIE, vol 7245, pp 724519Google Scholar
  27. 27.
    Ding S, Su C, Yu J (2011) An optimizing BP neural network algorithm based on genetic algorithm. J Artif Intell Rev 36(2):153–162CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Physics and Optoelectronic EngineeringXidian UniversityXi’anChina
  2. 2.Dipartimento di InformaticaUniversità degli Studi di MilanoCremaItaly
  3. 3.School of Electronic EngineeringXidian UniversityXi’anChina
  4. 4.Department of Embedded Systems EngineeringIncheon National UniversityIncheonKorea

Personalised recommendations