3D Image Sharpening by Grid Warping

Abstract

A method for sharpening of 3D volume images has been developed. The idea of the proposed algorithm is to transform the 3D neighborhood of the edge so that the neighboring pixels move closer to the edge, and then resample the image from the warped grid to the original pixel grid. The proposed technique preserves image textures while making the edges sharper. The effectiveness of the proposed method is demonstrated with synthetic volume images and real micro CT images.

The work was supported by Russian Science Foundation grant 14-11-00308.

1 Introduction

Image blur occurs in numerous types of 2D and 3D images, e.g. photographs, medical images of different modalities, telescopes, microscopes, satellite sensors, etc. As a consequence, the deblurring problem (also called deconvolution) has been widely investigated for the simpler 2D case and then extended to the 3D case. The deblurring methods require the complete knowledge or estimation of the blurring kernel, usually called as Point Spread Function (PSF). Image deconvolution is an ill-posed problem, and even the slightest error in the blur kernel may introduce strong artifacts in the resulting image.

Classical deblurring approaches are Wiener filtering, regularized linear least square algorithms, maximum likelihood and nonlinear iterative constrained methods like total variation regularization [1, 3, 13]. When the PSF is not known, blind deconvolution methods capable to identify the PSF and restore the image in the same time have been developed [5]. Inaccurate PSF estimation usually results in serious artifacts and distortions in the resulting image.

Blurred image enhancement methods that are not based on the PSF concept can be referred to as image sharpening methods. The commonly used method of image sharpening is the unsharp masking method [15, 16]. The main problem of the existing sharpening methods is unpleasant overshoot artifact and noise amplification that may appear in the output image [7].

To overcome the drawbacks of existing sharpening methods, an idea of grid warping was introduced. Instead of accurately estimating the blur kernel of the whole image and performing deconvolution, the pixel grid of the image is warped so that pixels in the neighborhood of the blurred edge move closer to the edge. This approach does not introduce artifacts like noise, overshooting, ringing effect because pixel values are not changed.

Warping approach for image enhancement was been introduced in [2]. The grid warping is performed according to the solution of a differential equation that is derived from the warping process constraints. Due to global nature of the method, the distortion of edges may appear.

In [8] the warping map is computed using simple local measures of the image. The measures are computed separately for rows and for columns of the image with restrictions that prevent two consecutive samples from interchanging their order in the 1D sequence. This approach does not introduce edge overshoot and does not amplify the noise, but it can introduce small local changes in the direction of edges.

In [18] a method is proposed that preserves the contours during enlargement, the method combines the warping of the coordinate point with the biasing of the signal amplitude.

The warping approach approach is related to the morphology-based sharpening [17] and shock filters [6, 14, 19]. These methods make the image appear piecewise constant which is effective mostly for cartoon-like images.

Special attention to edge model and preserving image textures was made in [9–11]. The 2D warping algorithm proposed in [10] has the following advantages:

• 1. It can be efficiently used as a post-processing method for global image deblurring methods.

• 2. No artifacts like ringing effect or noise amplification are introduced.

• 3. Unlike morphological methods and shock filters, the resulting images look natural and do not inevitably become piecewise constant.

• 4. It can be used as a standalone method of image sharpening. It is a good choice in the presence of strong noise and complex and non-uniform blur kernel.

In this work we propose an algorithm based on pixel grid warping [9–11] for edge sharpening of volume images.

2 Volume Image Blur Model

There are three main blur types: motion blur, defocus blur and camera sensor blur. The first blur type has irregular PSF which is hard to estimate while the latter two types have circular PSFs that can be approximated using Gauss filter kernel. Volume images are created for still objects so there is no motion blur and we can use Gauss filter as the approximation of real blur:

$$\begin{aligned} PSF(x, y, z) = \frac{1}{(2\pi )^{3/2}\sigma _x\sigma _y\sigma _z} \exp \left( -\frac{x^2}{2\sigma _x^2}-\frac{y^2}{2\sigma _y^2}-\frac{z^2}{2\sigma _z^2}\right) , \end{aligned}$$

(1)

where $\sigma _x, \sigma _y, \sigma _z$ are Gauss filter parameters that can be estimated using edge width estimation [12] by analyzing edge profiles corresponding to X, Y and Z axes.

Volume image dimensions usually have different resolution and different blur levels. Let $h_x, h_y, h_z$ be the pixel grid steps. To enable using grid warping algorithm, we need to make Gaussian blur uniform $\sigma _x = \sigma _y = \sigma _z$. It can be done by simple scaling pixel step values which leads to proportional change of corresponding Gauss filter parameters. So we assume the following PSF

$$\begin{aligned} PSF(x, y, z) = \frac{1}{(2\pi )^{3/2}\sigma ^3} \exp \left( -\frac{x^2 + y^2 + z^2}{2\sigma ^2}\right) . \end{aligned}$$

(2)

3 One-Dimensional Edge Sharpening

The idea of the proposed 3D image sharpening algorithm is taken from [10]. Consider one-dimensional edge profile g(x) centered at the point $x=0$ (see Fig. 1), d(x) — displacement function and $h(x) = g(x + d(x))$ — warped edge profile.

Fig. 1.

The idea of edge sharpening using grid warping. Red line is the original edge profile, blue line is the edge profile sharpened using grid warping (Color figure online).

Simple scaling $f(x) = kx$ will give sharper edge but it will also shrink the entire image. To make the edge sharper without changing image size, only the area near the edge center should be shrunk while the area outside the edge should be stretched proportionally.

The warped grid should remain monotonic (i.e. for any $x_1<x_2$ new coordinates should be $x_1 + d(x_1) \le x_2 + d(x_2)$), so the displacement function should match the following constraint:

$$\begin{aligned} d'(x) \ge -1. \end{aligned}$$

(3)

Another constraint localizes the area of warping effect near the edge center:

$$\begin{aligned} d(x) \rightarrow 0, \qquad |x| \rightarrow \infty . \end{aligned}$$

(4)

The displacement function d(x) greatly influences the result of the edge warping. On the one hand, the edge slope should become steeper. On the other hand, the area near the edge should not be stretched over some predefined limit to avoid wide gaps between adjacent pixels in the discrete case.

The work [10] constructs the displacement function d(x) using the proximity function

$$\begin{aligned} p(x) =&\ 1 + d'(x)\nonumber \\ d(x) = \int \limits _{-\infty }^{x}&{(p(y) - 1) dy}. \end{aligned}$$

(5)

The proximity is the distance between adjacent pixels after image warping. This value is inverse to the density value. If the proximity function p(x) is less than 1, then the image area is shrunk at the coordinate x. If the proximity is greater than 1, then the image is stretched. The identity transform has the value $p(x) \equiv 1$.

The constraint (3) leads to non-negativity of the proximity function. Also high values of the proximity function should be avoided to preserve image textures.

4 Volume Image Sharpening

For the problem of volume image sharpening, we approximate the proximity function (5) with the difference of two Gaussian functions in order to control the areas of shrinkage and stretching independently [10]:

$$\begin{aligned} p(x) =&\ 1 + \kappa (G_{\sigma _1}(x) - G_{\sigma _2}(x)), \quad \sigma _2 > \sigma _1,\nonumber \\&\quad \kappa = 1 / \left( G_{\sigma _1}(0) - G_{\sigma _2}(0)\right) . \end{aligned}$$

(6)

Parameter $\sigma _1$ controls the width of the densification area while parameter $\sigma _2$ controls the width of the rarefication area. We use $\sigma _1 = \sigma$ and $\sigma _2 = 2\sigma$. Fig. 2 illustrates using grid warping for edge sharpening using this proximity function.

Fig. 2.

Edge sharpening by grid warping using the proximity function (6). Blue function is the blurred edge with some noise. Red function is the result of warping. Green line is the proximity function (Color figure online).

To apply the warping algorithm to volume images, we adopt the two-dimensional extension [10] to the three-dimensional case. The displacement is a 3D vector field $\varvec{d}(x,y,z)$ with the following constraints similar to the 2D case:

• 1) The shapes of the edges cannot be warped, so $\varvec{d}(x_e,y_e,z_e)=0$ for each edge point $(x_e,y_e,z_e)$.

• 2) There cannot be any turbulence: $\mathrm {rot}~{\varvec{d}}=0$. Since $\mathrm {rot} \nabla u \equiv 0$, the displacement field is assumed to be gradient of some scalar function u(x, y, z): $\varvec{d}(x, y, z) = \nabla u(x, y, z)$.

• 3) The constraint (3) takes the form

  $$\begin{aligned} \mathrm {div} \varvec{d} \ge -1 \end{aligned}$$

  (7)

  and the proximity function looks as

  $$\begin{aligned} p(x,y,z) = 1 + \mathrm {div} \varvec{d}(x,y,z). \end{aligned}$$

  (8)

Since $\mathrm {div} \nabla \equiv \varDelta$, where $\varDelta$ is a Laplacian, the warping problem can be posed as a Dirichlet problem for the Poisson equation in the area of the image:

$$\begin{aligned} \begin{aligned} {\left\{ \begin{array}{ll} \varDelta u &{}= p(x, y, z) - 1,\\ u(x,y,z) &{}= 0 \text { at image borders}. \end{array}\right. } \end{aligned} \end{aligned}$$

(9)

The second constraint here is the boundary condition: the displacements at image borders should be equal to zero.

In order to get the same results as in the 1D case and to keep the edge pixels unwarped, the proximity value should be equal to the 1D proximity function depending on the distance to the edge. However, the distance to the closest edge $\rho$ as an argument of the proximity function $p(\rho )$ is not efficient as it may produce gaps between close edges. Also it blurs edge ends.

We suggest the following method for calculating the proximity function:

$$\begin{aligned} p(x_0, y_0, z_0) = \frac{ \sum \limits _{(x, y, z) \in E(x_0, y_0, z_0)} p(x_n) G_\sigma (x_t) |\varvec{g}(x, y, z)| }{ \sum \limits _{(x, y, z) \in E(x_0, y_0, z_0)} |\varvec{g}(x, y, z)| } \end{aligned}$$

(10)

where $E(x_0, y_0, z_0)$ is the set of edge points in the neighborhood of $(x_0, y_0, z_0)$. The 3D edge point set is obtained using 3D Canny edge detector [4].

The value $x_n$ is the projection and the value $x_t$ is the length of the rejection of the vector $(x_0 - x, y_0 - y, z_0 - z)$ on the edge gradient vector $\varvec{g}(x, y, z)$.

The function $p(x_n)$ is the 1D proximity function, weighting function $G_\sigma (x_t)$ is Gauss function with standard deviation equal to the edge’s blur $\sigma$.

We solve the partial differential equation (9) using Fourier transform technique.

Fig. 3.

Application of the proposed method to synthetic ball image: (a) Slices; (b) Blurred images; (c) 3D unsharp mask result; (d) Sharpening using 2D warping applied to each slice separately; (e) Proposed 3D image warping algorithm.

Fig. 4.

Application of the proposed method to synthetic prism image: (a) Slices; (b) Blurred images; (c) 3D unsharp mask result; (d) Sharpening using 2D warping applied to each slice separately; (e) Proposed 3D image warping algorithm.

Fig. 5.

Application of the proposed method to real micro CT image: (a) Source image slices; (b) 3D unsharp mask; (c)Proposed 3D image warping algorithm.

5 Results

We demonstrate the effectiveness of the proposed image sharpening method with two synthetic images of ball and prism (see Fig. 3 and Fig. 4). These images were corrupted by additive uniform noise and blurred using Gauss filter with $\sigma = 2$ and then deblurred by 3D unsharp mask with the same $\sigma$ and $\alpha = 3$, 2D sharpening by image warping [10] applied to every slice and proposed 3D sharpening algorithm.

It can be seen that the proposed algorithm makes 3D edges sharper and does not produce any artifacts unlike unsharp mask. Two-dimensional algorithm was unable to improve the sharpness of edge surfaces that cannot be detected as edges using planar images. PSNR results for the synthetic images are shown in Table 1.

Figure 5 demonstrate the application of the proposed method to real micro CT image.

Table 1. PSNR results of the proposed 3D image sharpening method for the synthetic images

6 Conclusion

The two-dimensional warping approach for image sharpening has been effectively extended to the three-dimensional case. The proposed algorithm sharps even image edge corners that cannot be properly treated by independent application of two-dimensional sharpening methods to 2D slices. Like the 2D warping method, the developed 3D warping method enhances only image edges neighborhood and practically does not affect image textures.