No-Reference 3D Mesh Quality Assessment Based on Dihedral Angles Model and Support Vector Regression

Abstract

3D meshes are subject to various visual distortions during their transmission and geometrical processing. Several works have tried to evaluate the visual quality using either full reference or reduced reference approaches. However, these approaches require the presence of the reference mesh which is not available in such practical situations. In this paper, the main contribution lies in the design of a computational method to automatically predict the perceived mesh quality without reference and without knowing beforehand the distortion type. Following the no-reference (NR) quality assessment principle, the proposed method focuses only on the distorted mesh. Specifically, the dihedral angles are firstly computed as a surface roughness indexes and so a structural information descriptors. Then, a visual masking modulation is applied to this angles according to the main characteristics of the human visual system. The well known statistical Gamma model is used to fit the dihedral angles distribution. Finally, the estimated parameters of the model are learned to the support vector regression (SVR) in order to predict the quality score. Experimental results demonstrate the highly competitive performance of the proposed no-reference method relative to the most influential methods for mesh quality assessment.

1 Introduction

With any application domain, 3D meshes are usually subject to different geometric transformations. These operations introduce slight distortions on the 3D shape of the object that may alter the visual quality of the model. The Mesh Visual Quality (MVQ) assessment tries to identify how much the original model has been distorted.

Many approaches have been used to evaluate the visual quality of a distorted mesh. Root mean squared error RMS [1] and Hausdorff distance [2] use a simple similarity between the reference mesh and the distorted one. This kind of metrics generally fails to reflect the perceived visual quality since it computes a pure geometric distance neglecting the main operations of the human visual system (HVS) [3]. Several metrics use different perceptual principles for a better estimation of the perceived quality [4–6]. All the metrics cited above are full reference metrics, i.e. the reference content is fully available. Another type of metrics where only a part of the information is available called reduced reference. In this type, the visual quality is evaluated by comparing some features extracted from both the reference mesh and the distorted one [7–9].

Despite their suitability to several type of distortion, the main drawback of theses methods, is the non availability of the reference mesh in such practical situations. To remedy this problem, we propose a novel no-reference method for 3D mesh quality assessment. Our contribution is twofold. First, we extract features only from the distorted mesh by estimating the parameters from the dihedral angles distribution model. Then, we employ theses features in a learning framework to predict the objective quality score using the support vector regression [10].

Fig. 1.

Flowchart of the proposed no-reference mesh quality assessment method.

The reminder of this paper is organized as follows. In Sect. 2, we give a brief overview of our proposed no-reference quality assessment method as well as a description of the different steps including the visual masking modulation and the feature learning. Experimental results and comparisons are provided in Sect. 3. Finally, we draw in Sect. 4 some concluding remarks and perspectives.

2 The Proposed No-Reference Quality Assessment Method

An overview of the proposed no-reference mesh quality assessment method is shown in Fig. 1. As we have already mentioned, this method focus only on the distorted mesh to predict the quality score, to put it differently, the reference mesh is not involved in any step of the processing. Given a distorted mesh, we extract in the first step the dihedral angles performed by normals of adjacent triangular faces. Afterwards, a visual masking modulation is involved in order to take into consideration the masking effect [3], which is an important characteristic of the human visual system (HVS). The next step is to estimate statistical parameters of the extracted dihedral angles using the Gamma distribution, this step is very crucial to deal with the time complexity. Finally, we intend to use a regression method for the constructed feature learning, we employ in our work the support vector regression with different kernels in order to predict the objective quality score.

2.1 Dihedral Angles Extraction

The first step of our processing is to extract dihedral angles representing the structural aspect of the 3D mesh. Dihedral angles were used previously by several works in the context of measuring the distortion of 3D meshes. Corsini et al. [7] behold that the dihedral angle is strongly related to the surface roughness, therefore, in that work dihedral angles are only used to construct the concept of global roughness. In our work, we avoid this assumption by taking the dihedral angle as a relevant information which can be used to extract statistical parameters. These latter are used as an input features for the regression based learning step. The dihedral angle by definition is the angle formed by two normals $N_{1}$ and $N_{2}$ of two adjacent triangular faces. The dihedral angle $\varPhi$ is calculated by:

$$\begin{aligned} \varPhi = acos \left( \frac{N_{1} \cdot N_{2}}{norm(N_{1}) \times norm(N_{2})} \right) \end{aligned}$$

(1)

Where norm is the Euclidean distance. The angles vector is then obtained by concatenating all angles computed from the whole 3D mesh.

$$\begin{aligned} \varPhi _{i} = \left[ \varPhi _{1} , \varPhi _{2} , \varPhi _{3}, . . ., \varPhi _{n} \right] \end{aligned}$$

(2)

Where n is the number of neighborhoods in the mesh.

2.2 Visual Masking Effect Modulation

The visual masking effect is one of the most important characteristics of the human visual system HSV. In the context of 3D modeling, the concept of visual masking can be explained as the fact that the human perception cannot notice a small distortion located on a rough area, whereas human observers are able to detect distortion easily in smooth areas. To take into account the visual masking effect, we have to mask the distortion in rough regions. Using dihedral angles, roughness can be identified by extraction high magnitude dihedral angles. To reduce the high magnitude dihedral angles, we multiply the angles vector $\varPhi _{i}$ by a roughness weight function $RW_{i}$ defined as follows:

$$\begin{aligned} RW_{i}=\exp \left( -\frac{\varPhi _{i}}{2 \cdot \sigma ^{2}} \right) \end{aligned}$$

(3)

where $\sigma$ is the standard deviation of the dihedral angles vector. The visual masking modulation is then calculated as:

$$\begin{aligned} \varPhi _{masking}= \varPhi _{i} \cdot RW_{i} \end{aligned}$$

(4)

2.3 Statistical Parameters Estimation

In order to restrict the amount of the learning data, and hence to keep the execution time complexity under control, we intend to use a statistical parameter estimation by a non-Gaussian statistical model which is the Gamma distribution. Figure 2 shows an example of histograms of Armadillo’s and Dinosaur’s angles. We observe that the estimated Gamma model fits perfectly the empirical dihedral angles distribution. Therefore, we can quite simply use the estimated parameters of the model and reduce the amount of data, and consequently optimize the computational time.

Fig. 2.

Histogram of the extracted dihedral angles from Armadillo (a) and Dinosaur (b) with noise addition, with their corresponding plots of the fitted Gamma distribution.

We say that a random variable x follows Gamma law with a shape parameter b and a scale parameter $\theta$ if it admits for probability density:

$$\begin{aligned} p(x;b,\theta ) = \frac{\theta ^{-b} x^{b-1}}{\varGamma (b)} \exp \left( -\frac{x}{\theta }\right) \> \> \> \> \> \> 0< x < \infty \end{aligned}$$

(5)

Where $\varGamma (.)$ denotes the Gamma function. We recall that parameters are estimated using the maximum likelihood ML method. The estimated parameters are then used as an input feature vectors for the support vector regression.

2.4 Feature Learning: Support Vector Regression (SVR)

Support Vector Regression (SVR) is an extension of the support vector machines (SVM) [10] for a numeric prediction. The support vector machine is a supervised classification system that finds the maximum margin hyperplane separating two classes of data. The training instances that are closest to this hyperplane are called support vectors. In order to predict the objective quality scores for distorted meshes, we use in our work the support vector regression. We denote by $x_{i}$ the feature vector for a distorted mesh $M_{i}^{d}$ with a subjective score $y_{i}$. The regression function of an observation x to classify is given as follows:

$$\begin{aligned} f_{SVR} = \sum _{x_{i} \in V_{s}} \alpha _{i} y_{i} K(x_{i},x) +b , \end{aligned}$$

(6)

where $V_{S}$ are the support vectors, $(x_{i} , y_{i} )$ presents the training set and $\alpha$ denotes the Lagrange multipliers obtained in the minimization process. Furthermore, $K(x_{i},x)$ is the kernel function, in this work we compare four different kernels: linear, polynomial, radial basis function (RBF) and sigmoid. Table 1 shows the different kernels used with their mathematical equations and parameters.

Table 1. Kernels used for the support vectors regression.

Besides the kernel parameters cited above, SVR involves another parameter C called the penalty parameter of the error term. For each chosen kernel, it is required to select the best parameters values, this selection is very crucial and strongly affect the prediction results. Discussions about the different tests as well as the experimental results are analyzed in the next section.

3 Experimental Results

In this section we evaluate the effectiveness and forcefulness of the proposed no-reference quality assessment method on two publicly available databases:

The LIRIS/EPFL General-Purpose Database¹ [4]: This database was created at the EPFL, Switzerland. It contains 4 reference meshes: Armadillo, Dyno, Venus and RockerArm, and 84 distorted models (88 models total). Two types of distortion are applied: smoothing and noise addition either locally or globally on the reference mesh Fig. 3 shows some models from the LIRIS/EPFL general-purpose database and their distorted versions. The subjective evaluation was done by 12 observers.

Fig. 3.

Some models from the LIRIS/EPFL general-purpose database and their distorted versions: (a) reference meshes. (b) distorted meshes, from left to right are respectively: Armadillo with global noise, Rockerarm with noise in smooth regions, Dinosaur with global smoothing, and Venus with noise in rough regions.

The LIRIS Masking Database (see footnote 1) [11]: This database was created at the Universite of Lyon, France. It contains 4 reference meshes: Armadillo, Bimba, Dyno and Lion, and 24 distorted models (28 model total). The local noise addition is the only type of distortion applied Fig. 4 shows some models from the LIRIS Masking Database and their distorted versions. The specific objective of this database is to test the capability of mesh visual quality metrics in capturing the visual masking effect. The subjective evaluation was done by 11 observers.

Fig. 4.

Some models from the LIRIS Masking Database and their distorted versions: (a) reference meshes. (b) distorted meshes with noise addition.

As we have mentioned before, we used in this work four different kernels for the support vector regression: Linear, polynomial, radial basis function (RBF) and sigmoid kernel, with a grid search space on the kernel parameters in order to select the best parameters values. The correlation between the perceptual distances produced by the method and the mean opinion scores (MOS) produced by subjects is used as criteria to evaluate the performance of an objective mesh visual quality MVQ metric. Usually two types of correlation coefficients are commonly used: the Pearson linear correlation coefficient which employed to measure the prediction accuracy, and the Spearman rank-order correlation coefficient which employed to measure the prediction monotonicity [12].

3.1 Comparison of Different Kernels

It is not evident beforehand which kernel to use to predict the quality score by the SVR, thus we compare four kernels: Linear, polynomial, RBF and sigmoid. The focal point is to select the adequate kernel with a selective parameters that conducts to the best SVR prediction on the two databases. Table 2 shows the tested kernels as well as the Pearson and Spearman correlation coefficients from the proposed method on the LIRIS masking database. Note that we determine the best kernel and parameters by maximizing jointly the $r_{s}$ and $r_{p}$ values.

Table 2. Correlation coefficients $r_{s}$ (%) and $r_{p}$ (%) of our proposed metric according to the tested kernels on the LIRIS masking database

According to Table 2, the experimental tests show that the correlation scores varies from a kernel to another, although there is not a huge difference between the predicted scores by the different kernels, but it is quite noteworthy that the RBF kernel with the pair $(C=2^{11},\gamma =2^{-15})$ shows a great performance and overtake the 90 % correlation threshold in term of both $r_{s}$ and $r_{p}$ coefficients. Similarly, we find that the RBF kernel gives the best results on the general-purpose database with a correlation scores $r_{s}=81.5\,\%$ and $r_{p}=87.5\,\%$ with the pair $(C=2^{-4},\gamma =2^{-5})$. Accordingly, we select the RBF as the most suitable SVR kernel used for the feature learning step.

3.2 Comparison with Full Reference and Reduced Reference Methods

In this section, we compare our proposed no-reference mesh quality assessment method with several existing full reference metrics as well as reduced reference metrics. Values of $r_{s}$ and $r_{p}$ from the compared objective mesh visual quality metrics on the two considered databases are listed in Tables 3 and 4. The proposed no-reference metric has the highest $r_{s}$ and $r_{p}$ values on the LIRIS masking database (whole corpus). In fact this database is manufactured to evaluate the visual masking effect. The good results and performances provided by the proposed method confirm that the visual masking modulation used in this work is very effective. Furthermore, the proposed method provides competitive scores on the general-purpose database (comparative scores with MSDM2 [4], TPDM [6] and FMPD [8]). These results show the effectiveness and forcefulness of the proposed no-reference quality assessment method.

Table 3. Correlation coefficients $r_{s}$ (%) and $r_{p}$ (%) of different objective metrics on LIRIS masking database.

Table 4. Correlation coefficients $r_{s}$ (%) and $r_{p}$ (%) of different objective metrics on LIRIS/EPFL general-purpose database.

4 Conclusion

We have designed and implemented an efficient and effective method for a no-reference mesh quality assessment. Given only a distorted mesh, the proposed scheme extract dihedral angles as relevant information that describe the structural information. The extracted feature vector is then modulated with a visual masking to take into consideration the visual masking effect, which is an important characteristic of the human visual system. The obtained vector is then modeled by the Gamma distribution in order to construct feature vectors with only 2 parameters instead of using whole features values. This step is very crucial to lessen the computational time. Once feature vectors are constructed, the proposed scheme predicts the numeric quality score by training features using the support vectors regression (SVR). Compared with existing full reference and reduced reference mesh quality assessment, the proposed no-reference method based on SVR provides more coherent results with subjective scores, especially on the LIRIS masking database, proving that the visual masking modulation used is very effective.

The current stage of development for the proposed method is limited to use only the visual masking effect, knowing that we can use more HVS characteristics. Extending the proposed method by using saturation effect will be a possible direction of future work. Another possible extension is to extract more relevant features that might present the visual aspect of the 3D mesh, and improve the current results.