Skip to main content
SpringerLink
Log in

Handling noise in textual image resolution enhancement using online and offline learned dictionaries

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

The resolution enhancement of textual images poses a significant challenge mainly in the presence of noise. The inherent difficulties are twofold. First is the reconstruction of an upscaled version of the input low-resolution image without amplifying the effect of noise. Second is the achievement of an improved visual image quality and a better OCR accuracy. Classically, the issue is addressed by the application of a denoising step used as a preprocessing or a post-processing to the magnification process. Starting by a denoising process could be more promising to avoid any magnified artifacts while proceeding otherwise. However, the state of the art underlines the limitations of denoising approaches faced with the low spatial resolution of textual images. Recently, sparse coding has attracted increasing interest due to its effectiveness in different reconstruction tasks. This study proves that the application of an efficient sparse coding-based denoising process followed by the magnification process can achieve good restoration results even if the input image is highly noisy. The main specificities of the proposed sparse coding-based framework are: (1) cascading denoising and magnification of each image patch, (2) the use of sparsity stemmed from the non-local self-similarity given in textual images and (3) the use of dual dictionary learning involving both online and offline dictionaries that are selected adaptively for each local region of the input degraded image to recover its corresponding noise-free high-resolution version. Extensive experiments on synthetic and real low-resolution noisy textual images are carried out to validate visually and quantitatively the effectiveness of the proposed system. Promising results, in terms of image visual quality as well as character recognition rates, are achieved when compared it with the state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. www.freetype.org.

  2. ppi: Pixel Per Inch.

  3. http://www.irfanview.com.

  4. http://www.benvista.com/photozoompro.

  5. https://code.google.com/p/tesseract-ocr.

  6. http://onlineocr.net/.

References

  1. Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006). https://doi.org/10.1109/tsp.2006.881199

    Article  MATH  Google Scholar 

  2. Alaei, A., Conte, D., Blumenstein, M., Raveaux, R.: Document image quality assessment based on texture similarity index. In: Proceedings of 12th IAPR Workshop on Document Analysis Systems (DAS), Santorini, Greece, pp. 132–137 (2016). https://doi.org/10.1109/DAS.2016.33

  3. Alaei, A., Conte, D., Raveaux, R.: Document image quality assessment based on improved gradient magnitude similarity deviation. In: Proceedings of 13th International Conference on Document Analysis and Recognition (ICDAR), Nancy, France, pp. 176–180 (2015). https://doi.org/10.1109/ICDAR.2015.7333747

  4. Banerjee, J., Jawahar, C.V.: Super-resolution of text images using edge-directed tangent field. In: Proceedings of DAS, pp. 76–83. Washington, DC, USA (2008). https://doi.org/10.1109/DAS.2008.26

  5. Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Proceedings of CVPR, pp. 60–65 (2005). https://doi.org/10.1109/CVPR.2005.38

  6. Caner, G., Haritaoglu, I.: Shape-dna: effective character restoration and enhancement for Arabic text documents. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 2053–2056. Washington, DC, USA (2010). https://doi.org/10.1109/ICPR.2010.506

  7. Chatterjee, P., Milanfar, P.: Clustering-based denoising with locally learned dictionaries. IEEE Trans. Image Process. 18(7), 1438–1451 (2009). https://doi.org/10.1109/TIP.2009.2018575

    Article  MathSciNet  MATH  Google Scholar 

  8. Chen, G., Zhu, F., Ann Heng, P.: An efficient statistical method for image noise level estimation. In: The IEEE International Conference on Computer Vision (ICCV) (2015)

  9. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001). https://doi.org/10.1137/S003614450037906X

    Article  MathSciNet  MATH  Google Scholar 

  10. Chou, S.L., Yu, S.S.: Sorting qualities of handwritten Chinese characters for setting up a research database. In: International Conference on Document Analysis and Recognition, pp. 474–477 (1993)

  11. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.O.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007). https://doi.org/10.1109/TIP.2007.901238

    Article  MathSciNet  Google Scholar 

  12. Dai, S., Han, M., Wu, Y., Gong, Y.: Bilateral back-projection for single image super resolution. In: Proceedings of ICME, pp. 1039–1042 (2007). https://doi.org/10.1109/ICME.2007.4284831

  13. Dalley, G., Freeman, W.T., Marks, J.: Single-frame text super-resolution: a Bayesian approach. In: Proceedings of ICIP, pp. 3295–3298 (2004)

  14. Davis, G., Mallat, S., Avellaneda, M.: Adaptive greedy approximations. Constr. Approx. 13(1), 57–98 (1997). https://doi.org/10.1007/BF02678430

    Article  MathSciNet  MATH  Google Scholar 

  15. Dong, W., Zhang, L., Lukac, R., Shi, G.: Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans. Image Process. 22(4), 1382–1394 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  16. Dong, W., Zhang, L., Shi, G., Wu, X.: Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. 20(7), 1838–1857 (2011). https://doi.org/10.1109/TIP.2011.2108306

    Article  MathSciNet  MATH  Google Scholar 

  17. Drira, F., Lebourgeois, F.: Denoising textual images using local/non-local smoothing filters: a comparative study. In: Proceedings of ICFHR, pp. 521–526 (2012). https://doi.org/10.1109/ICFHR.2012.198

  18. Drira, F., Lebourgeois, F., Emptoz, H.: Document images restoration by a new tensor based diffusion process: application to the recognition of old printed documents. In: Proceedings of ICDAR, pp. 321–325 (2009)

  19. Drira, F., Lebourgeois, F., Emptoz, H.: A new pde-based approach for singularity-preserving regularization: application to degraded characters restoration. IJDAR 15(3), 183–212 (2012)

    Article  Google Scholar 

  20. Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006). https://doi.org/10.1109/TIP.2006.881969

    Article  MathSciNet  Google Scholar 

  21. Fadili, M.J., Starck, J.L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64–79 (2007)

    Article  Google Scholar 

  22. Freeman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. 28(3), 1–10 (2010)

    Google Scholar 

  23. Hale, C., Smith, E.H.B.: Human image preference and document degradation models. In: Proceedings of ICDAR, Curitiba, Brazil, pp. 257–261 (2007). https://doi.org/10.1109/ICDAR.2007.135

  24. Hartley, R.T., Crumpton, K.: Quality of OCR for degraded text images. In: ACM Conference on Digital Library, pp. 228–229 (1999)

  25. Hoang, T.V., Smith, E.H.B., Tabbone, S.: Sparsity-based edge noise removal from bilevel graphical document images. IJDAR 17(2), 161–179 (2014)

    Article  Google Scholar 

  26. Kulesh, V., Schaffer, K., Sethi, I.K., Schwartz, M.: Handwriting quality evaluation. In: Proceedings of Second International Conference on Advances in Pattern Recognition (ICAPR), Brazil, pp. 157–165 (2001)

  27. Kumar, D., Ramakrishnan, A.G.: Quad: quality assessment of documents. In: International Workshop on Camera-based Document Analysis and Recognition, pp. 79–84 (2011)

  28. Kumar, J., Chen, F., Doermann, D.S.: Sharpness estimation for document and scene images. In: Proceedings of ICPR, Tsukuba, Japan, pp. 3292–3295 (2012)

  29. Kumar, V., Bansal, A., Tulsiyan, G.H., Mishra, A., Namboodiri, A.M., Jawahar, C.V.: Sparse document image coding for restoration. In: Proceedings of ICDAR, pp. 713–717 (2013)

  30. Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Proceedings of Neural Information Processing Systems (NIPS), pp. 801–808 (2007)

  31. Lelore, T., Bouchara, F.: FAIR: a fast algorithm for document image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 2039–2048 (2013). https://doi.org/10.1109/TPAMI.2013.63

    Article  Google Scholar 

  32. Li, R., Zhang, Y.J.: A hybrid filter for the cancellation of mixed Gaussian noise and impulse noise. In: IEEE International Conference on Information Communications and Signal Processing, pp. 508–512 (2003)

  33. Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001). https://doi.org/10.1109/83.951537

    Article  Google Scholar 

  34. Liu, X., Tanaka, M., Okutomi, M.: Single-image noise level estimation for blind denoising. IEEE Trans. Image Process. 22(12), 5226–5237 (2013). https://doi.org/10.1109/TIP.2013.2283400

    Article  Google Scholar 

  35. Lopresti, D.P., Zhou, J., Nagy, G., Sarkar, P.: Spatial sampling effects in optical character recognition. In: Proceedings of ICDAR, pp. 309–314 (1995)

  36. Lu, H., Kot, A.C., Shi, Y.Q.: Distance-reciprocal distortion measure for binary document images. IEEE Signal Process. Lett. 11(2), 228–231 (2004). https://doi.org/10.1109/LSP.2003.821748

    Article  Google Scholar 

  37. Luong, H., Philips, W.: Robust reconstruction of low-resolution document images by exploiting repetitive character behaviour. IJDAR 11(1), 39–51 (2008). https://doi.org/10.1007/s10032-008-0068-2

    Article  Google Scholar 

  38. Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. In: IEEE Transactions on Image Processing, pp. 53–69. ITIP (2007)

  39. Mirkin, B.: Clustering for Data Mining: A Data Recovery Approach, 1st edn. Chapman and Hall/CRC, Boca Raton (2005)

    Book  MATH  Google Scholar 

  40. Namane, A., Sid-Ahmed, M.A.: Character scaling by contour method. IEEE Trans. Pattern Anal. Mach. Intell. 12(6), 600–606 (1990)

    Article  Google Scholar 

  41. Nayef, N., Chazalon, J., Gomez-Kramer, P., Ogier, J.M.: Efficient example-based super-resolution of single text images based on selective patch processing. In: Proceedings of DAS, pp. 227–231 (2014). https://doi.org/10.1109/DAS.2014.25

  42. Obafemi-Ajayi, T., Agam, G.: Character-based automated human perception quality assessment in document images. IEEE Trans. Syst. Man Cybern. Part A 42(3), 584–595 (2012). https://doi.org/10.1109/TSMCA.2011.2170417

    Article  Google Scholar 

  43. Obafemi-Ajayi, T., Agam, G., Frieder, O.: Evaluation of human perception of degradation in document images. In: Proceedings of Document Recognition and Retrieval XVII, part of the IS&T-SPIE Electronic Imaging Symposium, San Jose, CA, USA, p. 75340T (2010). https://doi.org/10.1117/12.838748

  44. Pan, J., Hu, Z., Su, Z., Yang, M.: Deblurring text images via L0-regularized intensity and gradient prior. In: Proceedings of CVPR, Columbus, OH, USA, pp. 2901–2908 (2014). https://doi.org/10.1109/CVPR.2014.371

  45. Pan, J., Hu, Z., Su, Z., Yang, M.: L\(_{\text{0 }}\)-regularized intensity and gradient prior for deblurring text images and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 342–355 (2017). https://doi.org/10.1109/TPAMI.2016.2551244

    Article  Google Scholar 

  46. Park, J., Kang, H., Lee, S.: Automatic quality measurement of gray-scale handwriting based on extended average entropy. In: Proceedings of ICPR, Barcelona, Spain, pp. 4426–4429 (2000). https://doi.org/10.1109/ICPR.2000.902949

  47. Pati, Y.C., Rezaiifar, R., Rezaiifar, Y.C.P.R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of the 27th Annual Asilomar Conference on Signals, Systems, and Computers, pp. 40–44 (1993)

  48. Peng, X., Cao, H., Subramanian, K., Prasad, R., Natarajan, P.: Automated image quality assessment for camera-captured OCR. In: Proceedings of ICIP, Brussels, Belgium, pp. 2621–2624 (2011). https://doi.org/10.1109/ICIP.2011.6116204

  49. Pham, T.A., Delalandre, M.: Effective decompression of JPEG document images. IEEE Trans. Image Process. 25(8), 3655–3670 (2016). https://doi.org/10.1109/TIP.2016.2576024

    Article  MathSciNet  Google Scholar 

  50. Portilla, J.: Full blind denoising through noise covariance estimation using Gaussian scale mixtures in the wavelet domain. In: International Conference on Image Processing, vol. 2, pp. 1217–1220 (2004). https://doi.org/10.1109/ICIP.2004.1419524

  51. Pratt, W.K.: Digital Image Processing, 3rd edn. Wiley, New York (1978)

    MATH  Google Scholar 

  52. Pratt, W.K.: Digital Image Processing: PIKS Inside, 3rd edn. Wiley, New York (2001)

    Book  MATH  Google Scholar 

  53. Rice, S.V.: Measuring the accuracy of page-reading systems. Ph.D. thesis, UNLV, Las Vegas (1996)

  54. Rubinstein, R., Bruckstein, A., Elad, M.: Dictionaries for sparse representation modeling. Proc. IEEE 98(6), 1045–1057 (2010). https://doi.org/10.1109/JPROC.2010.2040551

    Article  Google Scholar 

  55. Rusiñol, M., Chazalon, J., Ogier, J.: Combining focus measure operators to predict OCR accuracy in mobile-captured document images. In: Proceedings of DAS, Tours, France, pp. 181–185 (2014). https://doi.org/10.1109/DAS.2014.11

  56. Sarkar, P., Nagy, G., Zhou, J., Lopresti, D.P.: Spatial sampling of printed patterns. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 344–351 (1998)

    Article  Google Scholar 

  57. Smith, E.H.B., Andersen, T.L.: Text degradations and OCR training. In: Proceedings of ICDAR, Seoul, Korea, pp. 834–838 (2005). https://doi.org/10.1109/ICDAR.2005.226

  58. Thouin, P., Du, Y., Chang, C.I.: Low resolution expansion of gray scale text images using Gibbs–Markov random field model. In: Proceedings of Symposium on Document Image Understanding Technologies, Columbia, MD, pp. 41–47 (2001)

  59. Thouin, P.D., Chang, C.I.: A method for restoration of low-resolution document images. IJDAR 2(4), 200–210 (2000)

    Article  Google Scholar 

  60. Turkan, M.: Nouvelles méthodes de synthèse de texture; application à la prédiction et à l’inpainting d’images. Thesis, Université Rennes 1 (2011)

  61. Walha, R., Drira, F., Lebourgeois, F., Alimi, A.M.: Super-resolution of single text image by sparse representation. In: Proceedings of the workshop on Document Analysis and Recognition (DAR), pp. 22–29. ACM, New York, NY, USA (2012). https://doi.org/10.1145/2432553.2432558

  62. Walha, R., Drira, F., Lebourgeois, F., Alimi, A.M., Garcia, C.: Resolution enhancement of textual images: a survey of single image-based methods. IET Image Process. 10(4), 325–337 (2016). https://doi.org/10.1049/iet-ipr.2015.0334

    Article  Google Scholar 

  63. Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Multiple learned dictionaries based clustered sparse coding for the super-resolution of single text image. In: Proceedings of International Conference on Document Analysis and Recognition, ICDAR, pp. 484–488 (2013)

  64. Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Single textual image super-resolution using multiple learned dictionaries based sparse coding. In: Proceedings of International Conference on Image Analysis and Processing, ICIAP, Part 2, pp. 439–448 (2013)

  65. Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Histogram of structure tensors: application to pattern clustering. In: Proceedings of WSCG, Plzen, pp. 345–352 (2014)

  66. Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: A sparse coding based approach for the resolution enhancement and restoration of printed and handwritten textual images. In: Proceedings of ICFHR, Greece, pp. 696–701 (2014)

  67. Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Sparse coding with a coupled dictionary learning approach for textual image super-resolution. In: Proceedings of ICPR, Stockholm, pp. 4459–4464 (2014)

  68. Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Resolution enhancement of textual images via multiple coupled dictionaries and adaptive sparse representation selection. Int. J. Doc. Anal. Recognit. 18(1), 87–107 (2015). https://doi.org/10.1007/s10032-014-0235-6

    Article  Google Scholar 

  69. Walha, R., Drira, F., Lebourgeois, F., Garcia, C., Alimi, A.M.: Joint denoising and magnification of noisy low-resolution textual images. In: Proceedings of ICDAR, Nancy, France, pp. 871–875 (2015)

  70. Weickert, J.: A review of nonlinear diffusion filtering. In: Proceedings of International Conference on Scale-Space Theory in Computer Vision, pp. 3–28 (1997)

  71. Xu, J., Ye, P., Li, Q., Liu, Y., Doermann, D.S.: No-reference document image quality assessment based on high order image statistics. In: Proceedings of ICIP, Phoenix, AZ, USA, pp. 3289–3293 (2016). https://doi.org/10.1109/ICIP.2016.7532968

  72. Yan, Z., Lu, Y., Li, J.: Super resolution of text image by pruning outlier. Proc. ICONIP 7064, 649–656 (2011)

    Google Scholar 

  73. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010). https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  MATH  Google Scholar 

  74. Yang, M.C., Wang, Y.C.F.: A self-learning approach to single image super-resolution. IEEE Trans. Multimed. 15(3), 498–508 (2013)

    Article  MathSciNet  Google Scholar 

  75. Yang, S., Wang, M., Chen, Y., Sun, Y.: Single-image super-resolution reconstruction via learned geometric dictionaries and clustered sparse coding. IEEE Trans. Image Process. 21(9), 4016–4028 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  76. Ye, P., Doermann, D.S.: Document image quality assessment: a brief survey. In: Proceedings of ICDAR, Washington, DC, USA, pp. 723–727 (2013). https://doi.org/10.1109/ICDAR.2013.148

  77. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Proceedings of International conference on Curves and Surfaces, pp. 711–730 (2012). https://doi.org/10.1007/978-3-642-27413-847

  78. Zhang, M., Desrosiers, C., Zhang, C., Cheriet, M.: Effective document image deblurring via gradient histogram preservation. In: Proceedings of ICIP, Quebec City, QC, Canada, pp. 779–783 (2015). https://doi.org/10.1109/ICIP.2015.7350905

  79. Zheng, Y., Kang, X., Li, S., He, Y., Sun, J.: Real-time document image super-resolution by fast matting. In: Proceedings of DAS, pp. 232–236 (2014). https://doi.org/10.1109/DAS.2014.32

Download references

Author information

Authors and Affiliations

  1. REGIM-Lab, ENIS, University of Sfax, BP 1173, 3038, Sfax, Tunisia

    Rim Walha, Fadoua Drira & Adel M. Alimi

  2. LIRIS, UMR5205, CNRS, INSA-Lyon, University of Lyon, 69621, Lyon, France

    Frank Lebourgeois & Christophe Garcia

Authors
  1. Rim Walha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fadoua Drira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frank Lebourgeois
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christophe Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adel M. Alimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rim Walha.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Walha, R., Drira, F., Lebourgeois, F. et al. Handling noise in textual image resolution enhancement using online and offline learned dictionaries. IJDAR 21, 137–157 (2018). https://doi.org/10.1007/s10032-017-0294-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-017-0294-6

Keywords

Search

Navigation