# A de-noising method based on L0 gradient minimization and guided filter for ancient Chinese calligraphy works on steles

- 63 Downloads

**Part of the following topical collections:**

## Abstract

A clear stele image of ancient Chinese calligraphy pieces is very useful for studying ancient Chinese calligraphy. However, due to hundreds of or even thousands of years of natural or artificial damage on stele, images of ancient Chinese stele calligraphy works usually suffer from a large amount of image noise, and which usually leads to a poor visibility. To address this problem, in this paper, we propose a de-noising method based on L0 gradient minimization and guided filter. It consists of two main operations in sequence: First, L0 gradient minimization is utilized to obtain a random-noise free map, and then the random-noise free map is used as a guided image, and convoluted with its corresponding original noised stele image by a guided filter to obtain an edge preserved random-noise free image. Finally, the eight-connection region-based de-noising technique is followed to remove ant-like isolated blocks. Experiments demonstrate that the proposed method is superior to several recent published stele image de-noising techniques in terms of preserving the character structures.

## Keywords

L0 gradient minimization Guided filter Connection region De-noising Stele image enhancement## 1 Introduction

However, to the best of our knowledge, though there are many works on image de-noising [2, 3, 4], few works have focused on ancient Chinese calligraphy images. For this reason, a de-noising method for ancient Chinese calligraphy works on steles based on L0 gradient minimization and guided filter is proposed in this paper. We first utilize L0 gradient minimization to smooth the noised stele image. The goal of this step is to obtain random-noise free maps. Then, we take the random-noise free map as a guided image and convolute it with its corresponding original noised stele image via a guided filter. The goal of this step is to obtain an edge preserved random-noise free image. Finally, we used the eight-connection region-based de-noising technique as a tool to remove isolated blocks. Experiments show that our method is superior to several recent published stele image de-noising techniques in preserving characters’ structures.

- 1)
We put forward a L0 gradient minimization guided image filter for random-noise removal. It consists of two operation steps: First, to smooth a noised stele image by using L0 gradient minimization technique. Then, to compute an edge-preserved noise free image by convoluting the smoothed image with the input noised stele image.

- 2)
We employ the eight-connected domain technique for ant-like noise removal. The effectiveness of the proposed method is demonstrated by experimental results.

The remainder of this paper is structured as follows: Section 2 simply introduces the related works. Section 3 describes the proposed method in detail. Section 4 presents all the experimental results. Section 5 concludes the paper.

## 2 Related work

Interests regarding image preprocessing, including image de-noising [5] and image enhancement [6], especially on ancient Chinese calligraphy image enhancement [1, 2, 3, 4, 7, 8, 9, 10] have seen increasing in recent years; for instance, Zheng et.al. [1] presented a de-noising method for stele images using guided filter on the L channel. Wang et al. [7] used a threshold method to remove noise from a Chinese calligraphy image. In [3], an adaptive shock filter based on anisotropic diffusion is proposed. In [8], the Laplacian pyramid is combined with the fractional Fourier domain to extract the structures of characters with rotation angles at different scales. In [9], an anisotropic diffusion filter with the width information of different strokes was proposed for noise removal. In [10], structure characteristics of Chinese character strokes and run-length statistics were used for stele de-noising. In [11], an integration of multiple de-noising filters was proposed for removing block and line noise from stele images. In [12], KSVD dictionary learning was used for de-noising as well as for preserving the character structures.

Although there are many works on stele image processing, research on stele image de-nosing are still very limited. Solutions for ancient Chinese calligraphy image de-noising are still opened.

Recently, a piece wise filtering model termed L0 gradient minimization has been employed for image de-noising. By optimizing an energy function based on prior gradient sparsity, the L0 gradient minimization model shows a decent ability to perform image de-noising and a strong ability to preserve sharp features [13] in respect to traditional image de-noising method, such as Gaussian filter and shock filter. Similar works are also found in [14, 15].

Motivated by the aforementioned research, in this paper, we employ L0 gradient minimization as well as a guided filter for ancient stele calligraphy image de-noising.

## 3 Proposed method

### 3.1 Overview

### 3.2 Random-noise free map computation with L0 gradient minimization

The goal of the computation in this section is to obtain a random-noise free map. For this purpose, there are several filters that can be chosen, such as the Gaussian filter, shock filter, and L0 gradient minimization [13]. Results with these filters were nearly identical. However, with respect to Gaussian filter and shock filter, the L0 gradient minimization model shows a decent ability to perform image de-noising and a strong ability to preserve sharp features by optimizing an energy function based on prior gradient sparsity [13]. For what was mentioned above, in this section, we select the L0 gradient minimization as the tools for random-noise free map computation. Firstly, we decompose the input image into two parts using L0 gradient minimization: a base image and detailed image. The L0 norm of the gradient specifies the number of non-zero gradients [13] and drives the minimization to produce an output that is comparable to input around strong edges (as shown in Fig. 2 by the color blocks), which is based on the previous gradient.

*I*(

*x*,

*y*) be an input image and

*B*(

*x*,

*y*) be its corresponding base image, and

*D*(

*x*,

*y*) be its corresponding detail image. The gradients of

*I*(

*x*,

*y*) are denoted as ∇

*I*(

*x*,

*y*). Formally, we define the energy function of L0 gradient minimization as:

*γ*controls the level of detail in the final image and ∇

*I*(

*x*,

*y*) represents the number of non-zero gradient and is described as:

*x*,

*y*) on the original regularization term ∇

*I*(

*x*,

*y*) to speed up the process.

*x*,

*y*) denotes a binary map. Any technique that can obtain sharp edges can be used to get the binary map, such as applying the differential of Gaussian (DOG) filter on

*I*(

*x*,

*y*) with a threshold, or applying the Laplacian of Gaussian filter on

*I*(

*x*,

*y*) with a threshold. In this paper, we apply the differential of Gaussian (DOG) filter on

*I*(

*x*,

*y*) with a threshold to get the binary map. The threshold value is determined based on experiments.

*α*

_{x}(

*x*,

*y*) and

*α*

_{y}(

*x*,

*y*) are introduced, corresponding to\( \frac{\partial B\left(x,y\right)}{\partial x} \) and\( \frac{\partial B\left(x,y\right)}{\partial y} \), respectively. Equation (1) is reconstructed as:

*β*is used to regulate the difference between the dependent variables and its corresponding gradients. To address this minimization, an alternating minimization strategy is used, viz. We keep one of the variable sets unchanged to obtain another variable set. In our work, we keep

*α*

_{x}(

*x*,

*y*) and

*α*

_{y}(

*x*,

*y*) unchanged and use the below cost function to first obtain

*B*(

*x*,

*y*):

*B*(

*x*,

*y*) unchanged and through the following cost function obtain

*α*

_{x}(

*x*,

*y*) and

*α*

_{y}(

*x*,

*y*)

*H*(|

*α*

_{x}(

*x*,

*y*)| + |

*α*

_{y}(

*x*,

*y*)| ) is a binary function returning 1 when ∣

*α*

_{x}(

*x*,

*y*) ∣ + ∣

*α*

_{y}(

*x*,

*y*) ∣ ≠ 0; otherwise, it returns 0. By alternatively computing (5, 6), we obtain the final base image

*B*(

*x*,

*y*) and its corresponding detail image

*D*(

*x*,

*y*). Because all of the random-noise remains in the detail image

*D*(

*x*,

*y*), we take the final base image

*B*(

*x*,

*y*) as the random-noise free map. Figure 3 shows an example of random-noise removal results with the L0 gradient minimization operation. It is observed that the L0 gradient minimization smoothed image (Fig. 3b) contains little noise compared with the original image (Fig. 3a).

### 3.3 Recover of the Chinese character structure using a guided filter

After the L0 gradient minimization operation, we obtain a random-noise-free image, as shown in Fig. 3b. However, we find that some stroke edge details are removed as random noise by the L0 minimization operation, as indicated by the yellow box in Fig. 3. The guided filter is an edge-preserving filter [16]. Its goal is to smooth input images by calculating the content of the guidance image. Many studies shows that edges in an image after using guided filter will change differently. For step edges, it is still step edges after using guided filter, but their ranges become smaller, which means that the step edges become smoother after guided filter; For ridge edges, if the ridge edges with small size are unaffected by the other edges, their variances are close to 0, then the ridge edges will disappear and tend to the background; Valley edges will become larger than the input. From what was mentioned above, we can see that the guided filter has well preserving ability on image edges. Therefore, it can be used for image texture recover.

For what was mentioned above, to recover the over-smoothed stroke edge details, guided filter is employed in this section.

*I*and an input image

*I*

_{in}, the corresponding guided filtering output image\( {I}_{\mathrm{out}}^{\mathrm{guided}} \) is defined as:

*ω*

_{k}is a window centered at the pixel

*x*,

*α*

_{k}and

*b*

_{k}are constants in

*ω*

_{k}, respectively. These two parameters are determined as:

*μ*

_{k}indexes the average value of input image

*I*

_{in}in window

*ω*

_{k}, and \( {\upsigma}_k^2 \) is the variance of input image

*I*

_{in}in window

*ω*

_{k}.

*n*

_{ω}is the number of pixels in window

*ω*

_{k}. \( {\overline{p}}_k \) represents the mean of the guided image in window

*ω*

_{k}.

*ε*represents the regularization parameter which is used to determine the changing of pixels’ intensity. The initial value of parameter

*ε*in Eq. (9) is determined by referring [16], and its finial optimized value is determined by experiment.

### 3.4 Ant-like blocks removal using the eight-connected region technique

From Fig. 4, we also find that although most of the noise in the input image is removed after L0 gradient minimization filtering, there remains a few isolated ant-like blocks in the smoothed images, as denoted by the red circle in Fig. 4c.

*x*+ 1,

*y*), (

*x*− 1,

*y*), (

*x*,

*y*+ 1), and (

*x*,

*y*− 1); pixels on the diagonal could not be taken as an edge though they are serve as an effective insulator between two pixel set, and as a result, this creates undesirable topological anomalies, whereas the eight-connected region technique consider a pixel as connected not only pixels on the same row or column, but also the diagonal pixels. For this reason, also inspired by the work described [10], we utilize the eight-connected region technique [10] in this section to address the ant-like blocks problem. The processing to remove ant-like blocks using eight-connected region technique consists of the following three basic steps:

- 1)
Compute the connected components using an eight-connected domain-based search technique;

- 2)
Calculate each component’s area first. Then, sort all of connected components in terms of their area from large to small. Many studies index that if the area value of a block is smaller than the area value of 2/3 of the array of permutations, then the block could be taken as an isolated noise component and could be removed from an image. For this reason, we take the area value of 2/3 of the array of permutations as the value of threshold T;

- 3)
Remove all of isolated components when the area is smaller than the value of threshold T.

## 4 Experimental results and discussions

### 4.1 Experimental setting

To the best of our knowledge, there is no standard ancient Chinese stele calligraphy image set available for benchmarking currently. Therefore, we collected 100 stele calligraphy images from the released image set by [1]. We also generated 50 synthetically noised stele images using Adobe Photoshop CS6 to quantitatively evaluate the proposed method.

For comparison, several state of the art image de-noising methods, including Zheng’s method [1], Zhang’s method [10], L0 gradient minimization smoothing [13], and block-matching 3D filtering (BM3D) [17], as well as methods in our two prior works [11, 12] are employed in our experiments. The results of these methods are produced in MatLab with authors suggested parameter settings. All experiments are performed on a PC with 4G RAM and a 2.60 GHz Intel Dual Core processor using MatLab R2015.

### 4.2 Qualitative results

### 4.3 Quantitative evaluation

In this section, we quantitative evaluate the performance of our method on synthetic and real noised stele images, respectively.

*I*is a noise-free image,

*K*is its noisy approximation, and MSE is the mean squared error (MSE) of

*I*and

*K*.

where *μ*_{I} is the average of *I*, *μ*_{K} is the average of *K*, \( {\sigma}_I^2 \) is the variance of *I*, \( {\sigma}_K^2 \)is the variance of *K*, *σ*_{I, K} is the covariance of *I* and *K*, *c*_{1} and *c*_{2} are two variables to stabilize the division with weak denominator.

Qualitative evaluation of different methods on synthetic noisy stele images

## 5 Conclusions

We put forward a de-noising method based on L0 gradient minimization and guided filtering method for ancient Chinese stele calligraphy image de-noising in this paper. Experiments by comparing our presented method with several state of the art de-noising methods show the superior of our method for stele image de-noising.

In our next work, we will focus on how to restore spoiled character structures for the reason of preserving the beauty of art work.

## Notes

### Acknowledgements

The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.

### Funding

Not applicable.

### Availability of data and materials

Please contact author for data requests.

### Authors’ contributions

All authors take part in the discussion of the work described in this paper. All authors read and approved the final manuscript.

### Authors’ information

Feihang Ge received the B.E. degree and the M.S. degree in computer science and technology from Xi’an University of Technology in 2002 and 2005, respectively. He is currently studying for his Ph. D Degree at Aichi Prefectural University, Aichi, Japan. His research interests include intelligent image processing and computer vision. E-mail: fhge2018@163.com

Lifeng He received the B.E. degree from Northwest Institute of Light Industry, Shaanxi, China, in 1982, the second B.E. degree from Xian Jiaotong University, Shaanxi, in 1986, and the M.S. and the Ph.D. degrees in artificial intelligence and computer science from Nagoya Institute of Technology, Aichi, Japan, in 1994 and 1997, respectively. He is currently a Professor at Aichi Prefectural University, Aichi, and a Guest Professor at Shaanxi University of Science and Technology, Shaanxi. From September 2006 to May 2007, he was a Research Associate at The University of Chicago. His research interests include intelligent image processing, computer vision, medical image processing. E-mail: helifeng@ist.aichi-pu.ac.jp

### Competing interests

The authors declare that they have no competing interests.

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## References

- 1.X. Zheng, Q. Miao, Z. Shi, Y. Fan, W. Shui, A new artistic information extraction method with multi channels and guided filters for calligraphy works. Multimedia Tools & Applications
**75**(14), 8719–8744 (2015). https://doi.org/10.1007/s11042-015-2788-x CrossRefGoogle Scholar - 2.J. Chen, X. Kang, Y. Liu, Z. Wang, Median filtering forensics based on convolutional neural networks. IEEE Signal Processing Letters
**22**(11), 1849–1853 (2015). https://doi.org/10.1109/LSP.2015.2438008 CrossRefGoogle Scholar - 3.K. Guemri, F. Drira, in
*Proc. of 6th International Conference of Soft Computing and Pattern Recognition, Tunis, Tunisia*. Adaptative shock filter for image characters enhancement and denoising (2015), pp. 279–283Google Scholar - 4.H. Li, C.Y. Suen, A novel non-local means image denoising method based on grey theory. Pattern Recogn.
**49**(1), 237–248 (2016). https://doi.org/10.1016/j.patcog.2015.05.028 CrossRefGoogle Scholar - 5.Y. B, An improved infrared image processing method based on adaptive threshold denoising. EURASIP Journal on Image and Video Processing
**2019**, 5 (2019) https://doi.org/10.1186/s13640-018-0401-8 CrossRefGoogle Scholar - 6.Z. Shi, M. Zhu, B. Guo, M. Zhao, Nighttime low illumination image enhancement with single image using bright/dark channel prior. EURASIP Journal on Image and Video Processing
**2018**, 13 (2018) https://doi.org/10.1186/s13640-018-0251-4 CrossRefGoogle Scholar - 7.S.Z. Wang, H.J. Lee, in
*Proc. of 6th International Conference on Document Analysis and Recognition, Seattle, WA, USA*. Dual-binarization and anisotropic diffusion of Chinese characters in calligraphy documents (2001), pp. 271–275Google Scholar - 8.H.T. Nguyen, N. Linh-Trung, in
*Proc. of 10th International Conference on Information Sciences Signal Processing and their Applications, Kuala Lumpur, Malaysia*. The Laplacian pyramid with rational scaling factors and application on image denoising (2010), pp. 468–471Google Scholar - 9.T. Koga, E. Uchino, N. Suetake, G. Hashimoto, T. Hiro, M. Matsuzaki, in
*Proc. of the 7th Conference on Circuits, Systems, Electronics, Control and Signal Processing, Tenerife, Canary Islands, Spain*. Edge-preserved smoothing method with special reference to intravascular ultrasound image using anisotropic diffusion filter controlled by weighted separability measure (2008), pp. 87–92Google Scholar - 10.J. Zhang, J. Yu, G. Mao, and X. Ye, Denoising of Chinese calligraphy tablet images based on run-length statistics and structure characteristic of character strokes, Journal of Zhejiang University SCIENCE A, vol.7, no.7, pp.1178–1186, July. 2006, https://doi.org/10.1631/jzus.2006.A
- 11.Z. Shi, B. Xu, X. Zheng, M. Zhao, An integrated method for ancient Chinese tablet images de-noising based on the assemble of image smooth and noise reduction. Multimedia Tools & Applications
**75**(19), 12245–12261 (2016). https://doi.org/10.1007/s11042-016-3421-3 CrossRefGoogle Scholar - 12.Z. Shi, B. Xu, X. Zheng, and M. Zhao, A KSVD based Chinese character structure preserved denoising method” Multimedia Tools & Applications, vol.76, no.13, pp.14921U˝ 14936, 2017, https://doi.org/10.1007/s11042-016-4284-3
- 13.L. Xu, C. Lu, Y. Xu, J. Jia, Image smoothing via L0 gradient minimization. ACM Trans. Graph.
**30**(6), 1U˝ 12 (2011). https://doi.org/10.1145/2024156.2024208 CrossRefGoogle Scholar - 14.Z. Jin, G. Zhou, D. Gao, Y. Zhang, EEG classification using sparse Bayesian extreme learning machine for brain–computer interface. Neural Comput. & Applic. (2018). https://doi.org/10.1007/s00521-018-3735-3
- 15.Y. Zhang, C. Nam, G. Zhou, J. Jin, X. Wang, A. Cichocki, in
*IEEE Transactions on Cybernetics*. Temporally constrained sparse group spatial patterns for motor imagery BCI (2018), pp. 1–11. https://doi.org/10.1109/TCYB.2018.2841847 CrossRefGoogle Scholar - 16.K. He, J. Sun, X. Tang, in
*Computer Vision- ECCV 2010. ECCV 2010*, ed. by K. Daniilidis, P. Maragos, N. Paragios. Guided image filtering (Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2010), pp. 1–14Google Scholar - 17.C.A. Metzler, A. Maleki, R.G. Baraniuk, in
*Proc. of 2015 IEEE International Conference on Image Processing, Quebec City, QC, Canada*. BM3D-AMP: a new image recovery algorithm based on BM3D denoising (2015), pp. 3116–3120Google Scholar - 18.A. Hore, D. Ziou, in
*Proc.of 20th International Conference on Pattern Recognition, Istanbul, Turkey*. Image quality metrics: PSNR vs. SSIM (2010), pp. 2366–2369Google Scholar - 19.Z. Wang, A.C. Bovik, Universal image quality index. IEEE Signal Processing Letters
**9**(3), 81–84 (2002)CrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.