Thermal Image Approximation Using BSpline Surfaces
 195 Downloads
Abstract
This article presents a method of filtering thermal images by means of approximation using cubic Bspline patch surfaces. Approximation uses chord length parameterization with nonuniform distribution of knots, depending on the change rate of pixel values in rows and columns of the image. Advantages of approximation with such parameterization are compared with popular uniform parameterization. Presented method is very versatile and can be used for example in 3D surface topography of objects tested with stereoscopic microscopes, in resolution change of arbitrary images or modeling processes depending on two parameters. Filtering results are presented using plugin for the Soft4Image program, which was designed to unify different digital signal processing algorithms.
Keywords
3D Bsplines Image approximation Nonuniform parameterization Topography1 Introduction
Many data and measurement points are acquired in the form of grids and images. For filtering data in such domain, surface approximation with splines is very attractive. Splines have local control ability, surfaces are limited by control point’s convex hull, and they are easy to differentiate. In recent years, a number of works were devoted to approximation using different kinds of Bsplines: classical Bsplines [1, 2], triangular Bsplines [3], Tsplines [4, 5], manifold splines [6], box and simplex splines [7, 8] and others. The NURBS (nonuniform rational Bsplines) curves and surfaces have gained considerable popularity in graphics [9]. By adding weights to base functions, they are even wider class of splines, allowing modeling all sections of the cone (ellipses, hyperbolas and parabolas). Quality of approximation depends on surface definition, placement of knots, control points and corresponding to them, parameter values. Chordal parameter values give fourth approximation order for cubic interpolation, but for higher degrees, more accurate parameterizations, for example with arc length of cubic polynomial between approximated points, give better results [10]. Approximation can also be improved upon when using as an estimation for curve length not segment lengths between points, but results from previous iteration of approximation attempt.
The common approach to Bspline surface fitting is using least squares or weighted least squares approximation. If the parameters are computed from the data points and the knots positions from these parameters, problem can be solved using system of linear equations [1]. Many researches, in order to achieve better approximation, treat parameters, knots and control points as variables. This leads to a nonlinear least squares problem, and authors are forced to utilize more sophisticated but also more computationally demanding, iterative algorithms. Ülker and Arslan [11] developed method based on artificial immune system, Xie and Qin used conjugate gradient method [12], and Gálvez and Iglesias utilized particle swarm optimization in [13]. Many authors focus on adaptive knot placement [14, 15, 16, 17]. Park and Lee proposed an approach that selects dominant parameter points to generate knots for Bspline curve fitting [18] and dominant columns for surfaces approximation [19].
Methods of artificial intelligence are more often used in problems of approximating irregular cloud of points, where triangulation of data (usually Delunay’s) is needed.
2 BSplines
The values of the t parameter for consecutive points \( C_{i} \) are nondecreasing and can be interpreted as the time after which the object moving along the curve reaches given point.
The parameters \( t^{*} \) defining the scope of the base functions are called knots. Since each base function is defined in d of the previous knots and one new, knots define connection points of the polynomials forming spline. Because of sharing the knots by the base functions, the vector \( t^{*} \) has the size of b + d + 1. Compact support determines local influence of the position of de Boor point \( P_{i} \) on a curve in the interval \( [t_{i}^{*} ,t_{i + d + 1}^{*} ) \). It also means that the position of the curve point depends only on \( d + 1 \) control points. Due to the limited support of the base functions, curve is contained in a convex hull of d + 1 control points. The sum of d + 1 weighted base functions is sufficient to determine the position of subsequent points on the curve (de Boor algorithm).

uniform—parameters of consecutive points C are equidistant and take values of indices:
$$ t_{i} = i $$(5) 
chordal parameterization—the distance between parameters is proportional to the length of the curve between points C. In practice, curve is estimated with straight lines between points C:
$$ t_{i} = \mathop \sum \limits_{k = 1}^{i} \left\ {C_{k}  C_{k  1} } \right\ $$(6) 
centripetal parameterization—defined similarly to chordal, but the distance is measured as a square root of the Euclidean distance. With interpolation, it does not create loops and cusps, running closer to the control polygon.
$$ t_{i} = \sqrt {\mathop \sum \limits_{k = 1}^{i} \left\ {C_{k}  C_{k  1} } \right\} $$(7)
In this way, the choice of parameterization for the placement of knots on the curve is preserved, e.g., for chordal parameterization, knots will be distributed evenly along the curve. This parameterization guarantees that every knot span contains at least one parameter \( t_{i} \) (Schoenberg–Whitney condition) and covariance matrix \( \varvec{N}^{\varvec{T}} \varvec{N} \) is positive definite and wellconditioned [20]. The same strategy can be used to determine curve points between approximated points after evaluation of control points (for example to draw the curve). Also, for the approximation of the function \( f(x) \) using chordal parameterization, it is often important to maintain even quantized \( x \)coordinate values. In case where no resolution change is made, this strategy ensures that the curve points are evaluated for exactly the same parameters as data points.
Each row of matrix C defines data point to be approximated. Lengths of vectors between these points are approximately proportional to the length of the curve between them and allow estimating values of the parameters t for the approximated points C in chordal parameterization.
Worth noting is the large computational cost of the covariance matrix \( \varvec{N}^{\varvec{T}} \varvec{N} \) for the number of basis functions close to the number of approximated points (when approaching interpolation). Calculations can be accelerated twice using symmetry of the covariance matrix.
3 Global Surface Approximation
Surface created from basis functions in the u direction with de Boor’s points lying on the Bspline curve in the v direction would give the same surface that one obtained from base functions in the v direction with control points lying on the curve in the u direction if not for different knot parameterizations in directions u and v (the factors in (16) are interchangeable). A common approach for parameterization of regular grids, like images, is to use uniform parameterization. This can easily lead to overshooting problems and lower approximation quality. Uniform curve parameterization provides always only second order of accuracy, while chordal parameterization gives degree + 1 approximation order for degrees less than 4 [10]. In our implementation, we treated rows and columns of data as points in multidimensional space and calculated chordal parameterization using lengths of segments between these points. This approach is consistent with calculating data point’s parameters for parametric curves from rows of matrix C as in (11).
Expressions (18) and (19) for the final surface are identical, but matrices \( \varvec{N}_{\varvec{v}} \) differ in both cases, because they depend on parameterization of matrix \( \varvec{C} \) columns and \( \varvec{S}_{{\varvec{cols}}} \) rows, respectively. Also, computational cost of tensor surface is lower, because matrix \( \varvec{C} \) is smaller (n × b) than \( \varvec{S}_{{\varvec{cols}}} \) (m × n).
The assumption of the same knots positions for all rows and similarly for columns is very strong and affects greatly quality of approximation. The process referred as lofting (term shared from ship’s hull designers) consists of constructing parallel curves approximating each column of the grid. Each column is approximated using its own parameters and knot vector. Family of these curves create approximated image in the column direction. After approximating columns, surface is processed in the same manner along rows. Algorithm is simple and gives much better results, but is numerically expensive and timeconsuming.
4 Results of Experiments
Thermographic images, due to lower resolution, higher background noise level and slower response time to thermal excitation, are characterized by higher noise compared to ordinary images. The SNR ratio (and consequently also the figure of merit) of IR cameras can be digitally improved by approximating such images. Such processing allows for more precise recognition of thermal inhomogeneities in the tested samples. The image presented in the analytical forms as a combination of linear bicubic surfaces has a simpler representation that allows its compression. It is independent of the original sequence resolution and simplifies its processing. An example of such processing may be the determination of thermal parameters of materials using thermal tomography algorithms, in which, instead of processing all the pixels of the sequence, only simple approximation functions are analyzed.
RMS errors of surface approximation
Basis count b  RMS Error (1E−3)  

Uniform param.  Tensor product  Columns–rows  Rows–columns  Lofting  
5  62.4  61.1  61.6  60.7  53.6 
10  55.8  49.4  49.1  45.0  33.3 
15  44.0  33.6  33.3  30.3  18.8 
20  35.7  26.0  25.7  23.9  15.4 
25  30.3  20.0  19.9  18.9  13.6 
30  25.6  15.4  15.4  14.9  12.2 
50  14.0  10.6  10.7  10.7  9.5 
70  9.5  8.7  8.8  8.8  7.4 
90  7.6  7.4  7.5  7.5  5.7 
100  6.6  6.5  6.5  6.5  4.5 
Execution times
Basis count b  Time (ms)  

Uniform param.  Tensor product  Columns–rows  Rows–columns  Lofting  
5  21  23  42  43  634 
10  25  27  45  48  819 
15  30  31  50  51  1064 
20  38  41  60  61  1413 
25  44  46  67  67  1832 
30  53  55  79  79  2404 
50  118  120  154  156  6346 
70  230  234  271  274  14 028 
90  415  416  453  458  26 361 
100  533  536  569  566  34 861 
5 Conclusions
This article presented three methods of image approximation using Bspline surfaces. They differ in data point and knot placement when using nonuniform parameterization. For uniform parameter distribution, all methods give the same result. This parameterization does not take into account speeds of change of the curves in u and v directions (derivatives), and the results of experiments clearly show this weakness. Much better results, with minimal overhead in computation, can be achieved using chordal parameterization. Creating surface from tensor product of Bsplines proved to be the fastest method, with errors slightly worse than successive global approximation of columns and rows of the image. This method should be preferred, when approximating images. Lofting method needs improvement in reducing computational complexity, but it clearly sets the boundaries for achievable approximation errors for given number of basis functions.
Notes
Acknowledgments
Materials for the article were presented at 21st European Microelectronics and Packaging Systems Conference, Warsaw 2017.
References
 1.L. Piegl, W. Tiller, The NURBS Book (Springer, Berlin, 2012)zbMATHGoogle Scholar
 2.Y. Zhang, J. Cao, Z. Chen, X. Li, X.M. Zeng, Bspline surface fitting with knot position optimization. Comput. Graph 58, 73–83 (2016)CrossRefGoogle Scholar
 3.Y. He, H. Qin, Surface reconstruction with triangular Bsplines, in Proceedings of Geometric Modeling and Processing (IEEE, 2004), pp. 279–287Google Scholar
 4.J. Zheng, Y. Wang, H.S. Seah, Adaptive Tspline surface fitting to zmap models, in Proceedings of the 3rd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (ACM, 2005), pp. 405–411Google Scholar
 5.J. Wang, Z. Yang, L. Jin, J. Deng, F. Chen, Parallel and adaptive surface reconstruction based on implicit PHTsplines. Comput. Aided Geom. Des. 28, 463–474 (2011)MathSciNetCrossRefGoogle Scholar
 6.X. Gu, Y. He, H. Qin, Manifold splines. Graph. Models 68, 237–254 (2006)CrossRefGoogle Scholar
 7.J. Cao, X. Li, G. Wang, H. Qin, Surface reconstruction using bivariate simplex splines on Delaunay configurations. Comput. Graph. 33, 341–350 (2009)ADSCrossRefGoogle Scholar
 8.Y. Tan, J. Hua, H. Qin, Physically based modeling and simulation with dynamic spherical volumetric simplex splines. Comput.Aided Des. 42, 95–108 (2010)CrossRefGoogle Scholar
 9.G. Farin, NURB Curves and Surfaces (A. K Peters, Natick, 1999)zbMATHGoogle Scholar
 10.M.S. Floater, Chordal cubic spline interpolation is fourth order accurate. IMA J. Numer. Anal. 26, 25–33 (2006)MathSciNetCrossRefGoogle Scholar
 11.E. Ülker, A. Arslan, Automatic knot adjustment using an artificial immune system for Bspline curve approximation. Inf. Sci. 179, 1483–1494 (2009)CrossRefGoogle Scholar
 12.H. Xie, H. Qin, A novel optimization approach to the effective computation of NURBS knots. Int. J. Shape Model. 7, 199–227 (2001)CrossRefGoogle Scholar
 13.A. Gálvez, A. Iglesias, Particle swarm optimization for nonuniform rational Bspline surface reconstruction from clouds of 3D data points. Inf. Sci. 192, 174–192 (2012)CrossRefGoogle Scholar
 14.X. Zhao, C. Zhang, B. Yang, P. Li, Adaptive knot placement using a GMMbased continuous optimization algorithm in Bspline curve approximation. Comput.Aided Des. 43, 598–604 (2011)ADSCrossRefGoogle Scholar
 15.W. Li, S. Xu, G. Zhao, L.P. Goh, Adaptive knot placement in Bspline curve approximation. Comput. Aided Des. 37, 791–797 (2005)CrossRefGoogle Scholar
 16.Y. Yuan, N. Chen, S. Zhou, Adaptive Bspline knot selection using multiresolution basis set. IIE Trans. 45, 1263–1277 (2013)CrossRefGoogle Scholar
 17.X. He, L. Shen, Z. Shen, A dataadaptive knot selection scheme for fitting splines. IEEE Signal Proc. Lett. 8, 137–139 (2001)ADSCrossRefGoogle Scholar
 18.H. Park, J.H. Lee, Bspline curve fitting based on adaptive curve refinement using dominant points. Comput. Aided Des. 39, 439–451 (2007)CrossRefGoogle Scholar
 19.H. Park, Bspline surface fitting based on adaptive knot placement using dominant columns. Comput.Aided Des. 43, 258–264 (2011)CrossRefGoogle Scholar
 20.C. De Boor, A practical guide to splines, in Applied Mathematical Sciences, revised ed., vol. 27 (Springer, Berlin, 2001)Google Scholar
 21.P.J. Hartley, C.J. Judd, Parametrization of Bezier type Bspline curves and surfaces. Comput. Aided Des. 10, 130–134 (1978)CrossRefGoogle Scholar
 22.P.J. Hartley, C.J. Judd, Parametrization and shape of Bspline curves for CAD. Comput. Aided Des. 12, 235–238 (1980)CrossRefGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.