An efficient solution to sparse linear prediction analysis of speech

Linear prediction (LP) analysis is a ubiquitous analysis technique in current speech technology. The basis of LP analysis is the source-filter production model of speech. For voiced sounds in particular, the filter is assumed to be an all-pole linear filter and the source is considered to be a semi-periodic impulse train which is zero most of the times, i.e., the source is a sparse time series. LP analysis results in the estimation of the all-pole filter parameters representing the spectral shape of the vocal tract. The accuracy of this estimation can be evaluated by observing the extent in which the residuals (the prediction error) of the corresponding prediction filter resemble the hypothesized source of excitation [1] (a perfect impulse train in case of voiced speech). However, it is shown in [1] that even when the vocal tract filter follows an actual all-pole model, this criterion of goodness is not fulfilled by the classical minimum variance predictor. Despite the theoretic physical significance, such sparse representation forms the basis for many applications in speech technology. For instance, a class of efficient parametric speech coders are based on the search for a sparse excitation sequence feeding the LP synthesizer [2].

It is argued in [3] that the reason behind the failure of the classical method in providing such sparse representation is that it relies on the minimization of l₂-norm of prediction error. It is known that the l₂-norm criterion is highly sensitive to the outliers [4], i.e., the points having considerably larger norms of error. Hence, l₂-norm error minimization favors solutions with many small non-zero entries rather than the sparse solutions having the fewest possible non-zero entries [4]. Hence, l₂-norm is not an appropriate objective function for the problems where sparseness constraints are incorporated. Indeed, the ideal solution for sparse residual recovery is to directly minimize the cardinality of this vector, i.e., the l₀-norm of prediction error which yields a combinatorial optimization problem. Instead, to alleviate the exaggerative effect of l₂-norm criterion at points with large norms of error, it is usual to consider the minimization of l₁-norm as it puts less emphasis on outliers. l₁-norm can be regarded as a convex relaxation of the l₀-norm and its minimization problem can be re-casted into a linear program and solved by convex programming techniques [5].

The l₁-norm minimization of residuals is already proven to be beneficial for speech processing [6, 7, 8]. In [6], the stability issue of l₁-norm linear programming is addressed and a method is introduced for both having an intrinsically stable solution as well as keeping the computational cost down. The approach is based the Burg method for autoregressive parameters estimation using the least absolute forward-backward error.

In [7], the authors have compared the Burg method with their l₁-norm minimization method using the modern interior points method and shown that the sparseness is not preserved with the Burg method. Later, they have proposed a re-weighted l₁-norm minimization approach in [8], to enhance the sparsity of the residuals and to overcome the mismatch between l₀-norm minimization and l₁-norm minimization while keeping the problem solvable with convex programming tools. Initially the l₁-norm minimization problem is solved using the interior points method and then the resulted residuals are used iteratively, to re-weight the l₁-norm objective function such that less weight is given to the points having larger residual norms. The optimization problem is thus iteratively approaching the solution for the ideal l₀-norm objective function. We also mention that, an interesting review is made in [9, 10], on several solvers for the general problem of mixed l _p l₀-norm minimization in the context of piece-wise constant function approximation, which indeed their adaptation to the problem of sparse linear prediction analysis can be beneficial (particularly the stepwise jump penalization algorithm, which is shown to be highly efficient and reliable in detection of sparse events).

In this article, we propose a new and efficient solution to sparse LP analysis which is based on weighting of the l₂-norm objective function so as to maintain the computational tractability of the final optimization problem and to avoid the computational burden of convex programming. The weighting function plays the most important role in our solution in maintaining the sparsity of the resulting residuals. We first extract from the speech signal itself, the points having the potential of attaining largest norms of residuals (the glottal closure instants) and then we construct the weighting function such that the prediction error is relaxed on these points. Consequently, the weighted l₂-norm objective function can be minimized by the solution of normal equations of liner least squares problem. We show that our closed-form solution provides better sparseness properties compared to the l₁-norm minimization using the interior points method. Also, to show the usefulness of such sparse representation, we use the resulting prediction coefficients inside a multi-pulse excitation (MPE) coder and we show that the corresponding multi-pulse excitation source provides slightly better synthesis quality compared to the estimated excitation of the classical minimum variance synthesizer.

The article is organized as follows. In Section 2, we provide the general formulation of the LP analysis problem. In Section 3, we briefly review previous studies on sparse LP analysis and the numerical motivations behind them. We present our efficient solution in Section 4. In Section 5, the experimental results are presented and finally in Section 6, we draw our conclusion and perspectives.

The consideration of the vocal tract filter in the source-filter production model as an all-pole filter results in the well-known autoregressive model for the speech signal x(n):

(1)

where a _k are the prediction coefficients, K is the order of prediction filter and r(n) is the prediction error or the residual. In the ideal case, when the {a _k } coefficients are perfectly estimated and the production mechanism verifies the all-pole assumption, the residual should resemble the hypothesized excitation source. In case of voiced speech, it should be a perfect semi-periodic impulse train which is zero most of the times, i.e., it is a sparse time series. The linear prediction analysis problem of a frame of length N can be written in the general matrix form as the l _p -norm minimization of the residual vector r:

(2)

where a is a vector representing the set {a _k } and

and N₁ = 1 and N₂ = N + K (For n < 1 and n > N, we put x (n) = 0). The l _p -norm is defined as . Depending on the choice of p in Equation 2, the estimated linear prediction coefficients and the resulting residuals would possess different properties.

The ideal solution to the LP analysis problem of Equation 2 so as to retrieve the sparse excitation source of voiced sounds, is to directly minimize the number of non-zero elements of the residual vector, i.e., its cardinality or the so-called l₀-norm [11]. As this problem is an N P hard optimization problem [8], its relaxed but more tractable versions (p = 1,2) are the most widely used.

Setting p = 2 results in the classical minimum variance LP analysis problem. Although the latter suggests the highest computational efficiency, it is known that this solution can not provide the desired level of sparsity, even when the vocal tract filter is truly an all-pole filter [1]. It is known that l₂-norm has an exaggerative effect on the points having larger values of prediction error (the so-called outliers). Consequently, the minimizer puts much effort on forcing down the value of these outliers, with the cost of more non-zero elements. Hence, the resulting residuals are not as sparse as desired.

It is known that this exaggerative effect on the outliers is reduced with the use of l₁-norm and hence, its minimization could be a meliorative strategy w.r.t the minimum variance solution, in that the error on the outliers are less penalized [11]. The solution to the l₁-norm minimization is not as easy as the classical minimum variance LP analysis problem but it can be solved by recasting the minimization problem into a linear program [12] and then using convex optimization tools [5]. However, it is argued in [6] that linear programming l₁-norm minimization, suffers from stability and computational issues and instead, an efficient algorithm is introduced, based on a lattice filter structure in which the reflection coefficients are obtained using a Burg method with l₁ criterion and the robustness of the method is shown to be interesting for voiced sound analysis. However, it is shown in [7] that the l₁-norm Burg algorithms behaves somewhere in between the l₂-norm and the l₁-norm minimization. Instead, the authors have shown that enhanced sparsity level can be achieved using modern interior points method [5] of solving the linear program. They have shown interesting results of such analysis and have argued that the added computational burden is negligible considering the consequent simplifications (granted by such a sparse representation) in applications such as open and closed loop pitch analysis and algebraic excitation search.

An iteratively re-weighted l₁-norm minimization approach is consequently proposed by the same authors in [8] to enhance the sparsity of residuals, while keeping the problem solvable by convex techniques. The algorithm starts by plain l₁-norm minimization and then, iteratively, the resulting residuals are used to re-weight the l₁-norm cost function such that the points having larger residuals (outliers) are less penalized and the points having smaller residuals are penalized heavier. Hence, the optimizer encourages small values to become smaller while augmenting the amplitude of outliers [13].

The enhanced sparsity properties of the re-weighted l₁-norm solution compared to the l₁-norm minimization, and also the better performance of the l₁-norm criterion compared to l₂-norm criterion, can be explained numerically with the help of the graphical representation in Figure 1. There, the numerical effect of different residual values on l _p -norm cost functions is graphically depicted. It can be seen that the penalty on outliers is increasing with p. Indeed, as p→0 the penalty of the corresponding cost function on non-zero values approaches l₀-norm cost function (where any non-zero value is equally penalized and there is no penalization of larger values). This will force the minimization to include as many zeros as possible as their weight is zero. In case of the re-weighted l₁-norm solution [8], any residual is weighted by its inverse at each iteration and hence, the equal penalization of any non-zero value (as in l₀-norm criterion) is achieved. In other words, if a point has a very large (resp. very small) residual, it will be less (resp. much more) penalized in the next iteration and so, the sparsity is enhanced iteratively.

Open image in new window

We aim at developing an alternative and efficient optimization strategy which approximates the desired sparsity of the residuals. Our approach is based on the minimization of a weighted version of the l₂-norm criterion. The weighting function plays the key role in maintaining the sparsity of the residuals. Other than pure numerical motivations on de-emphasizing the exaggerative effect of l₂-norm on outliers (as discussed in Section 3), the design of this function is motivated by the physical production process of the speech signal. We extract from the speech signal itself, the points which are physically susceptible of attaining larger values of residuals and we construct the weighting function such that the error at those outliers is less penalized.

The outliers of LP residuals have an important physical interpretation as their time-pattern follows the pitch period of the speech signal. In other words, they follow the physical excitation source of the vocal tract system, which is a sparse sequence of glottal pulses separated by the pitch period. Indeed, the impulse-like nature of this excitation source is reflected as effective discontinuities in the residual signal [14]: when no significant excitation is presented at the input of the vocal tract system, its output is resonating freely according to the hypothesized all-pole model, and hence, it is maximally predictable by the parameters of the filter. On the other hand, the predictability is minimized when the significant excitations takes place and hence, the output signal would be under the influence of both the excitation source and the vocal tract filter. Consequently, LP residual contains clear peaks (outliers) around the instants of significant excitations of the vocal tract system. Hence, if we have a-priori knowledge about these instants we can use this knowledge to impose constraints on the LP analysis problem, so as to relax the prediction error at those points. By doing so, we ensure that if any enhancement is achieved in the sparsity level of residuals, it also corresponds to the physical sparse source of excitation.

The instants of significant excitations of vocal tract are called the Glottal closure instants (GCI) [14]. The detection of GCIs has gained significant attention recently as it founds many interesting applications in pitch-synchronous speech analysis. Many methods are developed for GCI detection in adverse environment (a comparative study is provided in [15]) and the physical significance of the detected GCIs is validated by comparing them to the electro Glotto graph signal. In this article, we use the recent robust SEDREAMS algorithm [15]^a. The weighting function is then constructed such that less emphasize is given to the GCI points, and hence the exaggerative effect on the outliers of the residuals is canceled. We can now proceed to formalize the proposed solution.

4.1 Optimization algorithm

We opt for l₂-norm cost function to preserve computational efficiency and then we cope with its exaggerative effect on outliers, by careful down-weighting of the cost function at those points. Formally, we define following optimization problem for the recovery of sparse residuals:

(3)

where w(·) is the weighting function. Once w(·) is properly defined, the solution to Equation 3 is straight-forward. Indeed, setting the derivative of the cost function to zero results in a set of normal equations which can be solved as in the classical l₂-norm approach:

(4)

while in our case, R=(W⊙X)X ^T , p=w⊙(X ^T x), ⊙ denotes the element-wise product of the two matrices and:

It is interesting to mention that our experiments show that as long as the smoothness of the w(·) is maintained the stability of the solution is preserved. Indeed, the special form of the input vector X in Equation 2, is the one used in autocorrelation formulation of LP analysis using l₂-norm minimization. It is proven that autocorrelation formulation always results in a minimum-phase estimate of the all-pole filter, even if the real vocal tract filter is not minimum phase [1]. As our formulation is similar to the autocorrelation formulation, we can fairly expect the same behavior (though we don’t have a theoretical proof). This is indeed beneficial, as having a non-minimum phase spectral estimate results in saturations during synthesis applications. Our experiments show that such saturation indeed never happens. This is an interesting advantage of our method compared to l₁-norm minimization methods which do not guaranty a minimum phase solution, unless if additional constraints are imposed to the problem [7].

4.2 The weighting function

The weighting function is expected to provide the lowest weights at the GCI points and to give equal weights (of one) to the remaining points. To put a smoothly decaying down-weighting around GCI points and to have a controllable region of tolerance around them, a natural choice is to use a Gaussian function. We thus define the final weighting function as:

(5)

where T _k k =1 … N_gci denotes the detected GCI points and g(·) is a Gaussian function ( The parameter σ allows the control of the width of the region of tolerance and κ allows the control of the amount of down-weighting on GCI locations. Figure 2 shows a frame of voiced sound along with the GCI points detected by the SEDREAMS algorithm and the weighting function of Equation 5. It can be seen that this weighting function puts the lowest weights around the GCI locations (i.e., the expected outliers) and equally weights the remaining points. Numerically speaking, the minimizer is free to pick the largest residual values for the outliers and it concentrates on minimizing the error on the remaining points (hence the sparsity is granted as explained in Section 3). This can also be explained with regard to physical production mechanism of the speech signal: as the coupling of excitation source and vocal tract filter is maximized on GCIs, such weighting function assists the minimizer to exclude the points on which the coupling is maximized and concentrate its effort on speech samples where the source contribution is minimized. Such decoupling is investigated in the context of Glottal volume velocity estimation by closed phase inverse filtering techniques [16]. There, the whole time interval on which the glottis is expected to be open is localized and discarded from the analysis frame. Consequently, these methods require the availability of both GCI and Glottal opening instants (GOI). However, the determination of GOIs is much more difficult than GCI detection [16]. Moreover, as the analysis window is strictly limited to the closed phase [17], another practical issue may arise: this time-frame might be too short (for high-pitched voices for instance) such that the analysis becomes ineffective [16].

Open image in new window

We first show the ability of our approach in retrieving sparse residuals for stationary voiced signals and also we show that it can provide a better estimation of the all-pole vocal-tract filter parameters. We then show how such sparse modeling can enhance the performance of a multi-pulse excitation estimation. All the results presented in this section are obtained using the following set of parameters for w(·): κ = 0 . 9 and σ = 50. The choice of the parameters was obtained using a small development set (of few voiced frames) taken from the TIMIT database [18].

5.1 Sparsity of residuals for voiced sounds

We compare the performance of our weighted-l₂-norm solution with that of the classic l₂-norm minimization and also the l₁-norm minimization via convex programming. For minimization of the l₁-norm, we use the publicly available l₁-magic toolbox [12] which uses the primal-dual interior points optimization [5]. Figure 3 shows the residuals obtained for all these different optimization strategies. It is clear that the weighted-l₂ and also l₁-norm criteria achieve higher level of sparsity compared to the classic l₂-norm criterion. Moreover, a closer look reveals that our weighted-l₂-norm solution shows better sparsity properties compared to the l₁-norm minimization: in the former, each positive peak of residuals is followed by a single negative peak (of almost the same amplitude) while for the latter, any positive peak is surrounded by two negative peaks of smaller (but yet significant) values.

Open image in new window

This comparison can be further formalized by using a quantitative measure of sparsity. There exists plenty of such measures on which a review is provided in [19]. Among them, we use the kurtosis, as it satisfies three of the most important properties that are intuitively expected from a measure of sparsity: scale invariance, rising tide and Robin Hood [19]. Kurtosis is a measure of peakedness of a distribution and higher values of kurtosis implies higher level of sparsity. Table 1 shows the kurtosis of the residuals obtained from the three optimization strategies, averaged over 20 randomly selected sentences of both male and female speakers taken from TIMIT database. From the table, it is clear the our method achieves the highest level of sparsity as it obtains the highest values of the kurtosis.

Table 1

Comparison of the sparsity levels

Method	kurtosis on the whole sentence	kurtosis onvoiced parts
l2-norm minimization	51.7	39.7
l1-norm minimization	81.9	65.9
weighted-l2-norm minimization	86.6	69.7

Quantitative comparison of the level of sparsity of different LP analysis strategies.

5.2 Estimation of the all-pole vocal-tract filter

We also investigate the ability of our method in estimation of the all-pole filter parameters. To do so, we generate a synthetic speech signal by exciting a known all-pole system with a periodic sequence of impulses (at known locations). We then estimate these parameters from the synthetic signal by LP analysis using our method and the classical l₂-norm method. Figure 4 shows the frequency response of the resulting estimates along with the frequency-domain representation of the synthetic excitation source. It can be seen that for the l₂-norm minimizer, there is a clear shift in the peaks of the estimated filter towards the harmonics of the excitation source. Specifically, the first spectral peak is shifted toward the forth harmonic of the excitation source. Indeed, the effort of l₂-norm minimizer in reducing great errors (the outliers due to the excitation source), has caused the estimated filter to be influenced by the excitation source. However, our weighted-l₂-norm minimization makes a very well estimation of the original all-pole filter and there is no shift in the spectral peaks. This shows that our method effectively decouples the contributions of the excitation source and the all-pole filter (as the source contribution is de-emphasized by the weighting function).

Open image in new window

We introduced a simple and efficient solution to the problem of sparse residual recovery of the speech signal. Our approach is based on minimization of weighted l₂-norm of the residuals. The l₂-norm was used to preserve the simplicity and the efficiency of the solution, while the weighting function was designed to circumvent l₂-norm’s exaggerative effect on larger residuals (the outliers). This is done by de-emphasizing the error on the GCIs where, by considering the physical production mechanism of the speech, we expect the outliers to occur. We showed that our methodology provides better sparsity properties compared to the complex and computationally demanding l₁-norm minimization via linear programming. Moreover, the method is interestingly immune to instability problems as opposed to l₁-norm minimization. Also, we showed that such intrinsically sparse representation can result in slightly better synthesis quality by sparse multi-pulse excitation of the synthesis filter in MPE-coding framework. The performance, the efficiency and the stability of the proposed solution show a promising potential in speech processing as its application can be further investigated in a variety of applications in the general framework of speech synthesis. This would be the subject of our future communications.

^aWe opted for the SEDREAMS so as to benefit from its proven reliability, in order to focus on proof-of-concept.