A Color Quantization Based on Vector Error Diffusion and Particle Swarm Optimization Considering Human Visibility

Abstract

In this paper, we propose a new color quantization method for generation of the color-reduced images. The proposed method employs a vector error diffusion (VED) method and a particle swarm optimization (PSO). VED method based on Floyd-Steinberg dithering is used for display of the color-reduced image. Furthermore, a color palette used in VED method is optimized by PSO. PSO generates the effective color palette with evaluating a human visibility of the color-reduced image on the display. The validity and the effectiveness of the proposed method are confirmed by some experiments.

1 Introduction

Electronic papers (e-paper) have become increasingly popular for general consumers [1]. The e-paper is a display technology, and has the combination of viewing characteristics and highly portability as well as those of the ordinary printed-paper. Recently, some display companies provide not only the monochrome e-papers, but also the color e-papers. In addition, the color e-papers can display images as well as texts. However, there are no full color e-papers in the present market. Furthermore, the power consumption of the color e-paper becomes large with increasing the number of colors to be displayed. It is thus preferable to display the color-reduced image for long-time using.

In order to generate the color-reduced image, color quantization methods are used generically. The color image quantization is an important issue in the fields of image processing and computer graphics. The popular color quantization methods are popularity algorithm (PA) [2], median cut algorithm (MCA) [2], k-means clustering (KMC) [3] and so on. Those methods generate a color palette containing K colors. After generating the color palette, each pixel value in the full color image is replaced into the quantized value in the color palette. PA chooses the K colors with the highest frequencies, and generates the color palette by using the chosen K colors. PA is very simple, but needs preprocessing, e.g. multilevel dithering, to decrease the number of colors in the original full color image. Further, the essential details in the original full color image cannot be preserved appropriately, because some colors with low frequencies are discarded. On the other hand, MCA and KMC are kinds of partitioning algorithms. MCA divides the color space of the original full color image into K rectangular boxes containing an equal number of pixels in the color space of the original full color image. In other words, MCA splits the color space repeatedly along the median into the rectangular boxes until the desired number of colors is obtained. MCA uses the average color in each rectangular box as one of the colors in the color palette. KMC divides the color space of the original image into K Voronoi spaces (partitions) by using cluster centers. The cluster centers are adjusted so as to minimize the accumulated distance between the cluster center and each pixel value in the Voronoi partition. After adjusting, the K cluster centers are used as the color palette. MCA and KMC can realize certain levels of quantization, when these methods apply to the color quantization in the image. Furthermore, their modified methods, e.g. center cut algorithm [4], RWM-cut algorithm [5, 6], fuzzy C-means clustering [7] and so on, have been proposed in order to improve the quantization errors. In recent years, metaheuristic-based color quantization methods have also proposed [8, 9].

However, these methods generate pseudo edges, because the similar colors are merged into a certain color coercively. In order to suppress the generation of the pseudo edges, it is required to employ not only the color quantization method, but also the multilevel dithering method as the post-processing. Nevertheless, the color palettes generated by the conventional quantization methods are not suitable for the post-multilevel dithering process, because the conventional color quantization methods focus on only reducing the quantization error. Therefore, a novel color quantization framework is needed to prepare the suitable color palette for effective multilevel dithering.

In this paper, we propose a new color quantization method to improve a quality of the quantized image. In the proposed method, the suitable color palette is found by using a particle swarm optimization (PSO) [10]. PSO is a population-based stochastic optimization method based on swarm intelligence [11] such as a bacteria foraging optimization (BFO) and an artificial bee colony (ABC) algorithms, and it is becoming very popular due to its simplicity of its concept and implementation. The color palette found by PSO is used for the multilevel dithering. As the multilevel dithering method, we employ a vector error diffusion (VED) method [12]. In the search, PSO evaluates a quality of the output image generated by VED with the color palette, which is to be optimized. The evaluation is based on a mean squared error between the original full color image and the output image represented by the full color. In the transformation from the color-reduced output image to the full color output image, the Gaussian filter is employed to consider the visual characteristics of human simply.

The validity and the effectiveness of the proposed method are verified by applying it to some test images.

2 Conventional Color Quantization Methods

2.1 Median Cut Algorithm

In MCA, the full color space divided into K color spaces. In particular, MCA splits a rectangular box in the color space into two rectangular boxes containing equal number of pixels. The cutting plane is decided based on one of the coordinate axes with a largest range of image pixel values, and pass through the median point of the color distribution projected on this axis. The splitting operation is performed repeatedly until that the number of the divided boxes is equal to that of the reduced color, i.e., K colors. At the end of the above-mentioned operation, the average color in each rectangular box is employed as one of the colors in the color palette. In other words, the colors in each rectangular box are merged into the average color.

MCA as well as PA can be performed by a simple procedure. However, the essential details in the full color image cannot be preserved appropriately, because some colors with low frequencies are discarded. Furthermore, the volume in a low-density part of the color space can be very large. Therefore, the quantization error tends to be large.

2.2 K-means Clustering Algorithm

KMC is a very popular method for general clustering. In KMC, input vectors are divided into K groups (clusters). In KMC, a center of each cluster as the centroid of its member input vectors is calculated to find the minimum of a sum-of-squares cost function (quantization error) $\phi$ represented by:

$$\begin{aligned} \phi = \sum _{j=1}^{K}\sum _{\ell =1}^{N_{\chi (j)}} \Vert {{\varvec{u}}}_{j,\ell } - {{\varvec{p}}}(j) \Vert , \end{aligned}$$

(1)

where K is the number of clusters. $\chi (j)$ and $N_{\chi (j)}$ stand for j-th cluster and the number of members in $\chi (j)$, respectively. ${{\varvec{u}}}_{j,\ell }$ represents $\ell$-th member in $\chi (j)$. ${{\varvec{p}}}(j)$ is the center of mass of the members in $\chi (j)$, and is calculated by:

$$\begin{aligned} {{\varvec{p}}}(j) = \frac{1}{N_{\chi (j)}} \sum _{\ell =1}^{N_{\chi (j)}} {{\varvec{u}}}_{j, \ell }. \end{aligned}$$

(2)

KMC is performed by the following steps:

• Step 1: K cluster centers ${{\varvec{p}}}(1), {{\varvec{p}}}(2), \cdots {{\varvec{p}}}(K)$ are initialized by the randomly-selected input vectors.

• Step 2: The cluster membership for each input vector is assigned to the nearest cluster center.

• Step 3: The cost function represented by Eq. (1) is calculated.

• Step 4: The cluster centers are updated by Eq. (2).

• Step 5: For each input data and all clusters, Steps from 2 to 4 until all cluster centers converge.

3 Proposed Method

In this paper, we propose a new color quantization method to improve a quality of the reduced-color image. In the proposed method, the appropriate color palette used for a vector error diffusion (VED) process is found by a particle swarm optimization (PSO). In this section, we first describe about VED for multilevel dithering after quantizing the color. Then, a procedure of PSO-based color palette generation method in order to realize the fine VED is described.

3.1 Vector Error Diffusion

VED is one of the multilevel dithering methods. A full color image is transformed into the K-color image by applying VED. The color palette for VED is then represented by ${{\varvec{p}}}_{\small {\text {VED}}} = \{{{\varvec{p}}}(1), {{\varvec{p}}}(2), \cdots {{\varvec{p}}}(K) \}$. VED is performed by the following processes. First, the closest color ${{\varvec{p}}}(k^*)$ is chosen by:

$$\begin{aligned} k^*=\arg \min _k \Vert {{\varvec{p}}}(k) - {{\varvec{I}}}_e(n) \Vert , \end{aligned}$$

(3)

where ${{\varvec{I}}}_e(n)$ represents a sum of n-th pixel value ${{\varvec{I}}}(n)$ in the full color image and the accumulated errors from the neighboring pixels to n-th pixel. In the next step, the n-th pixel ${{\varvec{I}}}_o(n)$ in the output image is replaced by ${{\varvec{p}}}(k^*)$. Further, an error ${{\varvec{e}}}(n)$ of n-th pixel is calculated by:

$$\begin{aligned} {{\varvec{e}}}(n)={{\varvec{I}}}_e(n) - {{\varvec{I}}}_o(n). \end{aligned}$$

(4)

After calculating the error, the weighted error is diffused into the neighboring pixels that have not yet been processed as shown in Fig. 1 based on Floyd-Steinberg method. VED is performed for each pixel along with the horizontal scan line from upper left to lower right.

Fig. 1.

Diffusion coefficients (weights) with its directions in Floyd-Steinberg method. The cell denoted by ${{\varvec{e}}}(n)$ represents the pixel currently being processed.

Fig. 2.

A conceptual sketch of the proposed method

3.2 Generation of Color Palette by using Particle Swarm Optimization

In the proposed method, the appropriate color palette ${{\varvec{P}}}_{\small {\text {VED}}}$ for employing the above mentioned VED is found by PSO. Figure 2 shows a conceptual sketch of the proposed method. PSO is motivated from the simulation of simplified animal social behaviors such as fish schooling, bird flocking, etc. Further, PSO is becoming very popular due to its simplicity of its concept and implementation.

The ordinary PSO as well as other population-based optimization methods starts with the random initialization of a population of individuals (particles). The particles are updated by exchanging fitness information obtained from the environment with each other. The individuals of the population thus move towards better solutions. Each individual moves with a velocity in the D-dimensional searching space. The velocity is iteratively updated according to its own and the other particles’ flying experiences. The position of i-th particle and its fitness value at time step t are represented as ${{\varvec{x}}}_{i}(t)=(x_{i1}(t), x_{i2}(t),\cdots ,x_{iD}(t))$ and $f_i(t)$, respectively. The velocity, which corresponds to a rate of the position change, is represented as ${{\varvec{v}}}_{i}(t)=(v_{i1}(t),v_{i2}(t),\cdots ,v_{iD}(t))$ for i-th particle at time step t.

In the proposed method, ${{\varvec{x}}}_{i}(t)$, which is the combined color vectors in ${{\varvec{P}}}_{\small {\text {VED}}}$ (i.e., $D=3K$), is represented by:

$$\begin{aligned} {{\varvec{x}}}_{i}(t)=({{\varvec{p}}}(1), {{\varvec{p}}}(2), \cdots {{\varvec{p}}}(K)). \end{aligned}$$

(5)

As the initial values of ${{\varvec{x}}}_{i}(0)$, in order to realize the effective search, we employ the noise added color palette obtained by the conventional KMC. In the initialization, the Gaussian noises are added to each element of the particle in order to generate various initial particles similar to the color pallet obtained by the conventional KMC.

In this paper, the purpose of this study is to generate the color-reduced image, which is the effective for visual contact. However, it is inherently difficult to compare the quality of the color-reduced image with that of the full-color input image. Thus, it is required to quantitative evaluation considering the visual contact. In order to evaluate the quality of the color-reduced image with considering the visual characteristics of human, the squared error between the full-color input image and the weighted average of color-reduced image is often used [14]. we thus employ a mean squared error (MSE) as the fitness function defined by:

$$\begin{aligned} f_i(t)=\frac{1}{N}\sum _{n=1}^{N} \Vert {{\varvec{I}}}(n) - {{\varvec{I}}}'_{o, {{\varvec{x}}}_{i}(t)}(n) \Vert ^2, \end{aligned}$$

(6)

where $\Vert \cdot \Vert$ stands for the Euclidean norm. ${{\varvec{I}}}(n)$ represents the input image. ${{\varvec{I}}}'_{o, {{\varvec{x}}}_{i}(t)}(n)$ stands for the output full color image obtained by VED with the combined color vector ${{\varvec{x}}}_{i}(t)$, and it is calculated by:

$$\begin{aligned} {{\varvec{I}}}'_{o, {{\varvec{x}}}_{i}(t)}=G *{{\varvec{I}}}_{o, {{\varvec{x}}}_{i}(t)}, \end{aligned}$$

(7)

where G and “$*$” stand for a 2-dimensional Gaussian filter and the convolution operator, respectively. N is the number of pixels in the image. The Gaussian filter is employed in order to calculate the weighted average of the color-reduced output image simply.

In the search of the proposed method, PSO finds the solution, which minimizes the fitness function. The PSO in the proposed method evaluates a quality of the output image ${{\varvec{I}}}_{o, {{\varvec{x}}}_{i}(t)}(n)$ generated by VED with the color palette ${{\varvec{x}}}_{i}(t)$, i.e., ${{\varvec{P}}}_{\small {\text {VED}}}$. In the evaluation, a similarity between the input full color image ${{\varvec{I}}}(n)$ and the output full color image ${{\varvec{I}}}'_{o, {{\varvec{x}}}_{i}(t)}(n)$ are calculated. In the transformation from the color-reduced output image ${{\varvec{I}}}_o$ to the full color output image ${{\varvec{I}}}'_o(n)$, the Gaussian filter is convoluted to the color-reduced output image. In the searching process, the best position of the i-th particle from step 0 to step t is recorded and represented as ${{\varvec{b}}}_i(t)=(b_{i1}(t),b_{i2}(t),\cdots ,b_{iD}(t))$. ${{\varvec{b}}}_i(t)$ is frequently called personal best position (p-best). Furthermore, the best position in the population from step 0 to step t is represented as ${{\varvec{g}}}(t)=(g_1(t),g_2(t),\cdots ,g_D(t))$. ${{\varvec{g}}}(t)$ is frequently called global best position (g-best). In order to update each particle position, $\ell$-th element $v_{i\ell }(t+1)$ in the velocity ${{\varvec{v}}}_{i}(t+1)$ is calculated with its own current, p-best and g-best positions by:

$$\begin{aligned} v_{i\ell }(t+1) =w(t)\cdot v_{i\ell }(t) + c_1\cdot r_1 \cdot \left\{ b_{i\ell }(t)-x_{i\ell }(t) \right\} +c_2\cdot r_2\cdot \left\{ g_\ell (t)-x_{i\ell }(t) \right\} , \end{aligned}$$

(8)

where $\ell =1,2,\cdots ,D$. $r_1$ and $r_2$ are random values in the range [0, 1]. $c_{1}$ and $c_{2}$ are positive constants. $\ell$-th element of the position ${{\varvec{x}}}_{i}(t)$ of the particle is updated with Eq. (8) by:

$$\begin{aligned} x_{i\ell }(t+1)=x_{i\ell }(t)+v_{i\ell }(t+1). \end{aligned}$$

(9)

In PSO, each particle in the population shares the information on the global best position with each other. The particle stochastically thus moves by using information on the global and the personal best positions in the searching space. Furthermore, PSO can search the appropriate color palette, which generates the fine color-reduced output image.

4 Experimental Results

In order to verify the performance of the proposed method, it is applied to 8 test images in the standard image database (SIDBA). The size of each image is 256$\times$256.

In this experiment, the quality of the output image generated by the proposed method is compared to those of the output images generated by the conventional MCA and KMC. Furthermore, the performance of the proposed method is tested in cases where $K=32$ and 64. The conventional MCA has no parameter. In the conventional KMC, the number of iteration is fixed to 100, empirically. In the proposed method, PSO has some parameters. The number of particles is 20. The both parameters $c_1$ and $c_2$ are decided to 1.49, which is well-used value in various optimization problems. The weight in the updating and the number of iteration steps are fixed to 0.72 and 50, respectively. Furthermore the Gaussian noise added to each particle in the initialization is generated in the normal distribution N(0, 5). The output images in the conventional methods are generated by VED with the color palettes obtained by MCA and KMC. The window size and variance of the Gaussian filter used for generation of the full color output image are 3$\times$3 pixels and 0.5, respectively.

Table 1. Quantization error (QE) and mean squared error (MSE) after color quantization in case where $K=32$. Conv. Methods 1 and 2 are median cut algorithm (MCA) and k-means clustering (KMC), respectively.

Table 2. Quantization error (QE) and mean squared error (MSE) after color quantization in case where $K=64$. Conv. Methods 1 and 2 are median cut algorithm (MCA) and k-means clustering (KMC), respectively.

Table 1 shows the quantization and the mean squared errors after the color quantization by each method in case where $K=32$. As shown in Table 1, the quantization error of the conventional method 2 (KMC) is better than those of the conventional method 1 (MCA) and the proposed method. In other words, KMC can realize the finest color quantization. However, the proposed method can generate the finest output image comparing to the conventional methods from the perspective of the MSE. In addition, the proposed method gives the smaller MSE in case where it is applied to the test images that contain the intricate details and smooth parts. Furthermore, Table 2 shows the quantization and the mean squared errors after the color quantization by each method in case where $K=64$. The color quantization results in case where $K=64$ shown in Table 2 denote the same tendency of those in case where $K=32$.

Fig. 3.

Resulting images (Couple) in case where K=32

Fig. 4.

Resulting images (Parrots) in case where K=32

Fig. 5.

Resulting images (Sailboat) in case where K=64

Fig. 6.

Resulting images (Lenna) in case where K=64

Figures 3 and 4 show examples of the resulting images in case where $K=32$. In each figure (a), white boxes show the parts of remarkable improvements by using the proposed method. In Fig. 3, the output image by the proposed method gives natural impression better than that of the conventional method 2, although the MSE of the output image by the proposed method is almost same to that by the conventional method 2. In Fig. 4, there are some dithering artifacts in the output images by the conventional method 1 and 2. On the other hand, the output image by the proposed method has less dithering artifacts than the conventional methods, and can preserve the essential details of the original image. Moreover, it can be observed that the proposed method has high expression ability of intricate detail parts. Figures 5 and 6 show examples of the resulting images in case where $K=64$. The impressions of the output images in case where $K=64$ also denote the same tendency of those in case where $K=32$.

Through the experimental results, it can be said that the proposed method generates the fine color-reduced image with low dithering artifacts and pseudo edges. Furthermore, the proposed method has better keeping quality of the original image than the conventional color quantization methods. In the proposed method, the computational cost for executing the PSO is gradually high with increasing the numbers of particle and iteration steps. However, it is inherently difficult to execute some computer program in the e-paper. Thus, it is assumed that the proposed system is executed just one time on the ordinary computer before transferring the image data to the e-paper. Furthermore, the proposed system by using the modified PSO which realizes effective search and GPGPU techniques can be executed without any waiting time.

5 Conclusions

In this paper, we proposed a new color quantization method to improve the quality of quantized image. The proposed method employed VED and PSO for the color quantization. In the proposed method, the appropriate color palette for use of VED was found by using PSO considering human visibility approximated by the Gaussian filter. In the search, PSO evaluates not quantization error, but a quality of the output image. The evaluation is based on the MSE between the original full color image and the output image transformed by the Gaussian filter into the full color.

The validity and the effectiveness of the proposed method were confirmed by applying it to some test images. Future work is to evaluate the quality of the output image based on not only MSE, but also other image quality assessment metrics such as SSIM and FSIM.