Data compression for measured heterogeneous subsurface scattering via scattering profile blending
 759 Downloads
Abstract
Subsurface scattering involves the complicated behavior of light beneath the surfaces of translucent objects that includes scattering and absorption inside the object’s volume. Physically accurate numerical representation of subsurface scattering requires a large number of parameters because of the complex nature of this phenomenon. The large amount of data restricts the use of the data on memorylimited devices such as video game consoles and mobile phones. To address this problem, this paper proposes an efficient data compression method for heterogeneous subsurface scattering. The key insight of this study is that heterogeneous materials often comprise a limited number of base materials, and the size of the subsurface scattering data can be significantly reduced by parameterizing only a few base materials. In the proposed compression method, we represent the scattering property of a base material using a function referred to as the base scattering profile. A small subset of the base materials is assigned to each surface position, and the local scattering property near the position is described using a linear combination of the base scattering profiles in the log scale. The proposed method reduces the data by a factor of approximately 30 compared to a stateoftheart method, without significant loss of visual quality in the rendered graphics. In addition, the compressed data can also be used as bidirectional scattering surface reflectance distribution functions (BSSRDF) without incurring much computational overhead. These practical aspects of the proposed method also facilitate the use of higherresolution BSSRDFs in devices with large memory capacity.
Keywords
Subsurface scattering Data compression BSSRDF Mobiles1 Introduction
One of the important goals in the field of computer graphics is the realistic representation of the appearance of realworld objects. Among such objects, the representation of translucent objects such as human skin, leaves, and biological tissue is challenging because of subsurface scattering, which is a complex light interaction which occurs beneath the surface of a sample. Subsurface scattering is represented numerically by a vast number of scattering parameters, especially for heterogeneous translucent materials. Although these parameters can be measured by special setups which involve camera(s) and projector(s) [1, 2, 3], the large data sizes of the obtained representations often complicate their practical application, particularly in the case of realtime rendering on devices with limited random access and graphics memory, such as video game consoles, tablets, and mobile phones.
To address these problems, we propose a data compression method for BSSRDFs of heterogeneous translucent materials. The key insight of this study is that many nonbiological materials such as waxes, soaps, candles, and minerals are often comprised of a limited number of base materials. For example, the crosssection surface of the onyx shown in Fig. 1 is comprised of approximately 20–30 base materials. Based on this observation, we represent the scattering properties of heterogeneous translucent materials by parameterizing only a few base materials. Figure 1 represents a comparison between the proposed compression method and a representative previous method by Song et al. [5] in terms of the visual qualities of rendered graphics. As shown in this figure, the proposed compression method achieves nearly identical visual quality although the size of the output data using the proposed method is only 2.94% compared to that of the previous study.
In the proposed method, the base materials are indexed by 8bit unsigned integers, and the linear combination weights for the base scattering profiles are represented by 32bit floating point numbers. Therefore, the data size assigned to each pixel is \((8 + 32) \times K = 40K\). We empirically determined that \(K = 2\) is typically sufficient to compress the BSSRDF data without significant loss of visual quality in rendered graphics. In this case, it is possible to further reduce the data size of a single pixel to 48 bits, which is significantly less than the local scattering profile that was assigned to each pixel in the previous method. The resulting data size for the proposed method is discussed in further detail in Sect. 3.

it achieves an efficient representation for heterogeneous subsurface scattering that requires only 40K bits for each pixel when K base materials are assigned to the pixel,

the resulting data size can be determined flexibly by changing the value of K,

for example, with \(K = 2\), approximately 30 times smaller data size than that of the stateoftheart method is achieved,

resulting compressed data can be used in standard realtime rendering algorithms without incurring computational overhead or the loss of visual quality.
2 Related work
2.1 Subsurface scattering models
Light transport behavior in translucent materials is theoretically described according to the radiative transfer theory [7], and most of the practical approximations used in the computer graphics community are derived based on this theory. For homogeneous materials, Jensen et al. [8] proposed a practical dipole model derived from an analytical dipole diffusion approximation, and Donner and Jensen [9] extended the dipole model to multilayered materials, such as human skin. d’Eon and Irving [10] performed an elaborate analysis of subsurface light transport and introduced a computation model referred to as the quantized diffusion model. Frisvad et al. [11] proposed a directional dipole model to appropriately handle the directional effect of incident light. Recently, Frederickx and Dutré [12] introduced a variant of the dipole model that considers the relationship between the incident and outgoing directions induced by strong forward scattering.
For heterogeneous materials, most surfacebased representations are based on the BSSRDF [4]. The main advantage of the BSSRDF is that it does not explicitly represent material distribution in object volumes. A BSSRDF \(S(x_i, \omega _i; x_o, \omega _o)\) describes the ratio of exiting radiance at position \(x_o\) to direction \(\omega _o\) to incident radiant flux at \(x_i\) from direction \(\omega _i\). When BSSRDFs are used for heterogeneous materials, they are often discretized and compressed to provide greater flexibility in practical applications. For example, nonnegative matrix factorization [2] and local scattering profiles [5, 13] have been used to compress the discretized form of the functions. However, these representations focus mainly on physical accuracy rather than data size. Thus, the data size aspect of BSSRDF representations has not attracted much attention despite the longstanding research on BSSRDF representations.
The proposed data compression method also targets BSSRDFs just as the aforementioned previous studies. However, compared to the previous methods, we focus more on the final data sizes of the compressed BSSRDFs. The proposed model achieves nearly the same visual quality in rendering results with a data size approximately 30 times smaller compared to the previous methods.
2.2 Translucency rendering
Here, we briefly introduce realtime rendering algorithms for translucent materials. We focus primarily on how BSSRDFs or other numerical models for subsurface scattering are used in rendering. Three classes of rendering algorithms are introduced, i.e., volumetric, imagespace, and surfacebased methods.
Volumetric methods simulate the light diffusion process in material volumes. This simulation can be performed efficiently on graphics hardware. Stam et al. [14] simulated light transport in an optically thick material as a diffusion process in the volume of materials. Wang et al. [3] extended their procedure to heterogeneous translucent materials by solving the diffusion equation for tetrahedralized object volumes. A similar method with a data structure, which is known as the light propagation volume (LPV) [15], is often used in realtime applications. Although the LPV is typically used to represent weak scattering in aerosols, it has also been applied to render optically dense translucent materials [16, 17]. The main drawback of these volumetric approaches is that they often produce blurry results because of the utilization of volumetric parameter mappings on lowresolution discretized grids. To obtain accurate rendering results, the object volumes must be precisely divided into small grids. However, storing different scattering parameters for such small grids requires significant memory resources. Consequently, volumetric methods are not suitable for the use in memorylimited devices.
Imagespace methods compute the BSSRDF convolution of surface irradiance on texture space or screen space. Lensch et al. [18] divided subsurface light transport into local and global components and computed the local component for close point pairs using texturespace image filtering. Shah et al. [19] proposed an interesting approach based on imagespace splatting for homogeneous translucent materials. Chen et al. [20] later extended their approach to heterogeneous materials. Currently, methods that employ screenspace filtering are commonly used in realtime graphics applications. In these methods, diffuse reflectance components are initially calculated and stored in a Gbuffer. Then, the diffuse reflection components are blurred in the screen space [21, 22]. Although these methods of screenspace filtering are computationally efficient, and the resulting visual quality is reasonable, the filtering operation is usually based on empirically determined parameters. Therefore, these methods cannot be directly used to physically measured BSSRDFs. Surfacebased methods rely on BSSRDFs to represent light attenuation factors between pairs of surface positions. Dachsbacher and Stamminger [23] sampled virtual point lights (VPL) from an irradiance buffer of shadow maps and evaluated the appearance of objects by summing the contributions from VPLs. Rather than computing vertextovertex light transport, Mertens et al. [24] computed areatoarea light transport by clustering triangular meshes of the target object. Recently, more sophisticated approaches using screenspace splatting [19] and multiple irradiance buffers [25] have been proposed. However, the aforementioned methods are all based on dipole approximation [8], which is only applicable to homogeneous materials. For heterogeneous materials, BSSRDFs are typically approximated and compressed to improve the computational efficiency of graphics hardware. In previous studies, compressed BSSRDFs were rendered by implementing an accelerated BSSRDF evaluation method known as hierarchical integration [26] with graphics hardware [2, 5]. However, these studies reported that they achieved at most 2 frames per second (fps) using computer systems of 10 years ago. Therefore, this performance is considered to be far from realtime frame rates, even if the latest computers are used. Recently, Chen et al. [20] introduced a realtime multiresolution splatting algorithm using BSSRDFs and achieved more than 30 fps using computer systems of 5 years ago. Considering the improvement in computation power of graphics hardware, their method can achieve realtime frame rates of more than 60 fps using current technology.
The notations used in this paper
Symbol  Meaning 

M  # of Base materials that comprise a target object 
K  # of Base scattering profile assigned to each pixel 
\(S(x_i, \omega _i; x_o, \omega _o)\)  BSSRDF 
\(F_{dr}(x)\)  Fresnel diffuse reflectance at x 
\(F_{t}(x, \omega _i)\)  Fresnel transmission for x and \(\omega \) 
\(R_d(x_i, x_o)\)  Diffuse reflectance between \(x_i\) and \(x_o\) 
\(P_{x}(r)\)  Scattering profile at x in distance r 
\(\hat{P}_{x}(r)\)  Logscale scattering profile at x, \(\left( = \log P_{x}(r) \right) \) 
\(\hat{P}_{b}(r)\)  Logscale base scattering profile of base material b 
\(\hat{P}_{b}^{h}\)  hth coefficient of the logscale base scattering profile \(P_{b}\) 
\(b_{x, k}\)  kth base material assigned to x 
\(t_{x, k}\)  kth linear combination weight at x 
\(\varTheta _{x}\)  Set of parameters for x, i.e., \(\{ b_{x, k} \}_k\) and \(\{ t_{x, k} \}_k\) 
\(\mathscr {B}\)  Set of base materials 
a  Acceptance probability for updating base material assignments 
p  Progression ratio to increase the acceptance probability a 
\(N_\mathrm{iter}\)  # of Iteration for the proposed alternating optimization method 
3 BSSRDF compression
In the following explanation, we used the symbols listed in Table 1. We do not explicitly provide the definitions of these symbols for simplicity of explanation.
3.1 Background
3.2 Compression by scattering profile blending
As described in the previous subsection, the compression method proposed by Song et al. [5] assigns different local scattering profiles to different surface positions. However, this expression is obviously redundant when the target material is composed of a few base materials. For example, the crosssection surface of the onyx shown in Fig. 2 contains at most 20 base materials, whereas Song’s representation assigns different scattering profiles to its target area of 229 \(\times \) 229 pixels. Rather than the perpixel assignment of the local scattering profiles, the proposed compression method represents each local scattering profile using the scattering profiles of the base materials. The scattering profiles of the base materials are referred to as the base scattering profiles.
Figure 2 shows the compressed BSSRDF representation for the proposed method. According to the number of base materials assigned to a single pixel, we refer to the representation of KBMPP as an abbreviation of “K base materials per pixel.” The simplest representation with base scattering profiles is obtained by assigning a single base material profile to a single location (1BMPP). This can be extended by assigning two or more base scattering profiles to each location. To represent the local scattering profile at a given location, the weighted averages of the assigned base scattering profiles are taken in the log scale. As shown in Fig. 2, each pixel generally has data for M mixture ratios and M indices for the base materials. The 2BMPP model, which is a special case of the KBMPP model, can further reduce the resulting data size because two mixture ratios, i.e., \(t_1\) and \(t_2\), can be represented with a single t as \(t_1 = t\) and \(t_2 = 1  t\). Thus, only one illustration is shown for the mixture ratio of the 2BMPP model in Fig. 2.
3.3 Parameter optimization
The second partial optimization step is inserted after every iteration for these iterative nonlinear solvers, i.e., the augmented Lagrangian and LBFGSB methods. In the second step, we fix the profile coefficients \(\hat{P}_{b_{x, k}}^h\) and update the base material indices \(b_{x, k}\) and mixture ratios \(t_{x, k}\). To find optimal indices and mixture ratios, we need to solve an NPhard combinatorial optimization problem. Rather than solving this complex problem directly, we update the base material indices and mixture ratios approximately with a randomized process inspired by PatchMatch [6]. Let \(\varTheta _x\) be a set of parameters at pixel x that includes the base material indices \(\{ b_{x, k} \}\) and \(\{ t_{x, k} \}\). The parameter set \(\varTheta _x\) is temporarily substituted by the parameters \(\varTheta _{x'}\) of candidate pixel \(x'\). The parameters \(\varTheta _{x'}\) are accepted as the parameters of x provided that this substitution reduces the least square error in Eq. 5. In this updating process, the immediate neighbors of x to the left and above are used as candidate pixels in evennumbered iterations, and those to the right and bottom are used in oddnumbered iterations. We also use the random pixels chosen from the entire pixels as candidates. Unlike the original PatchMatch, we leverage a probabilistic update strategy to avoid unexpectedly updating parameters using the scattering profile coefficients that are not sufficiently converged. In the ith iteration, the parameters are updated based on the probability of \(\min (a_0 p^i, 1.0)\) even if the error values are reduced. Typically, we use \(a_0 = 0.1\) and set the value \(p = (1 / a_0)^{2 / N_\mathrm{iter}}\) such that \(a_0 p^{i} = 1\) when \(i = N_\mathrm{iter} / 2\). Figure 3 shows the update processes of the base material indices, mixture ratios, and reconstructed appearances in each iteration of the 2BMPP model. In this experiment, the number of total iterations \(N_\mathrm{iter} = 20\) and the indices and mixture ratios are only partially updated when the number of iteration was less than \(N_\mathrm{iter} / 2 = 10\).
In our implementation, we initialized the parameters with random values before performing the above alternating optimization process. The profile indices were chosen randomly for each pixel from \(\{ 1, \ldots , M \}\) such that any two of the base material indices did not overlap. Mixture ratios were all initialized with 1 / K, and the profile coefficients in the log scale were initialized with random real numbers between \(\,1.0\) and 1.0.
3.4 Realtime rendering
3.5 Estimated data size
Let N be the number of pixels in the target area, then the estimated data size of an uncompressed BSSRDF is (\(N^2\) pixel pairs) \(\times \) (4 bytes) \(\times \) (3 color channels) \(= 12 N^2\) bytes when each attenuation factor is represented with threechannel 32bit floating point numbers. If only the neighboring pixels are considered as the counterparts of the pairs, it is reduced to \(12NW^2\), where W is the window size for neighboring pixels. In the compression method by Song et al., a local scattering profile is assigned to each pixel in the target area. When each local scattering profile is defined by a piecewise linear function with S discrete coefficients (\(S \ll N\)), the data size is estimated as \(N \times S \times 4 \times 3 = 12NS\) bytes. Typically, the value of S is determined as \(S = (W  1) / 2\). In contrast, the proposed model only considers the profile functions of M base materials (\(M \ll N\)). Therefore, the data size for a base scattering profile is as large as 12MS bytes. In addition, we assign K pairs of 8bit unsigned integers and 32bit floating point number to a pixel, which corresponds to the base material indices and mixture ratios, respectively. The data size for these indices and mixture ratios is estimated as \((8 + 32) \times K \times N\) = 40KN bits, i.e., 5KN bytes. Overall, the estimated data size of the proposed compressed BSSRDF is \(12MS + 5KN\) bytes. In the case of \(K = 5\), the proposed model is more efficient than Song’s method if \(12MS + 25N < 12NS\), which can be simplified to \(N < 0.48 (N  M) S\). Because Song et al. reported that they typically used \(S = 25\) in their implementation, this condition can be rewritten as \(M < 0.92 N\). Consequently, the data size of the proposed compression method is smaller than that of the previous method in most cases. The advantage of the proposed method in terms of the data size is further improved when a large number of pixels N are included in the target area. In addition, for \(K = 2\), it is not necessary to use two floating point numbers for the mixture ratios of each pixel because two mixture ratios can be represented by either of them only. Therefore, the data size for the 2BMPP model can be as small as \(12MS + (8 \times 2 + 32) N / 8 = 12MS + 6N\) bytes.
3.6 Relationship to other data compression methods
Although our idea of using basis functions to compress the BSSRDF data is common with many other compression methods, the problem formulation introduced in this paper is much more complicated than that encountered in common approaches. The base scattering profiles are conceptionally similar to clusters with common clustering algorithms such as the kmeans and the fuzzy cmeans methods. In these methods, the assignment of a cluster to a single pixel does not affect neighboring pixels. Therefore, the problem of assigning the clusters to each pixel can be independently solved. However, in the BSSRDF compression problem that we solve as a part of the proposed solution, the assignment of the base scattering profile can affect neighboring pixels as well. This makes the optimization problem much more difficult to solve. Other compression methods such as principal component analysis for dimensionality reduction cannot be applied straightforwardly to our problem for the same reason.
Our problem formulation is also similar to that of the Gaussian mixture model in the sense that a target function is approximated with a mixture of multiple basis functions. The Gaussian mixture model approximates the original function using a basis function defined in the same domain. Therefore, to approximate the 8D BSSRDFs or 4D diffuse reflectance functions using the Gaussian mixture model, we need to use 8D or 4D basis functions. However, an accurate approximation of such higherdimensional function requires dense sampling of the function’s domain. Therefore, an approximation using the Gaussian mixture model is not reasonable to approximate highdimensional BSSRDFs.
Performance of our compression method and the comparison to the previous study by Song et al. [5]
Material  Art. stone  Chessboard  Onyx  Blue wax  Yellow wax  Jade 

Pixels  \(108 \times 108\)  \(222 \times 222\)  \(229 \times 229\)  \(228 \times 88\)  \(110 \times 110\)  \(260 \times 260\) 
Size (MB)  
Original  163  857  912  281  210  947 
Song et al. [5]  2.5  10.7  11.4  4.4  2.6  14.6 
Size (kB)  
1BMPP  39  80  83  48  43  95 
2BMPP  98  326  345  148  103  433 
3BMPP  203  770  817  329  212  1041 
5BMPP  319  1263  1341  529  333  1717 
Time (s)  
1BMPP  3 m 45 s  23 m 10 s  23 m 12 s  6 m 54 s  5 m 17 s  24 m 23 s 
2BMPP  4 m 07 s  24 m 04 s  21 m 48 s  7 m 25 s  4 m 57 s  22 m 56 s 
3BMPP  5 h 45 m  33 h 21 m  35 h 19 m  9 h 50 m  7 h 14 m  38 h 08 m 
5BMPP  8 h 15 m  46 h 41 m  20 h 01 m  13 h 49 m  10 h 21 m  53 h 48 m 
4 Results and discussion
We experimented with the compression and rendering system described in this paper using a computer with an Intel Core i76700K 4.0 GHz CPU, an NVIDIA GeForce GTX 1070 graphics card, and 32 GB RAM. The compression program was implemented using C++, and the realtime rendering method was implemented using C++ with OpenGL and GLSL as the graphics API. The BSSRDFs used in our experiments were physically measured using the previous method proposed by Peers et al. [2]. The experiments described in this report were performed using \(M = 128\) base materials unless otherwise specified. Owing to space limitations, all results for mixture ratios, base material indices, and reconstructed appearances for \(K = 1, 2, 3\), and 5 are provided in the supplementary materials.
4.1 Selecting the number K
In the proposed compression method, the number K is an important factor in determining the final data size, computation time for compression, and visual quality of the rendered images. The final data sizes and computation times for different K values are provided in Table 2. In addition, the appearance of the reconstructed material under different illumination and the relative errors of the diffuse albedo are compared in Fig. 5. Here, a diffuse albedo value \(\rho _d(x)\) for pixel x is computed by integrating \(R_d\) over the local scattering range A as \(\rho _d(x) = \int _{A} R_d(x, y) dy\), and the relative error is computed as \(\Vert \rho _d(x)  \rho '_d(x) \Vert / \Vert \rho _d(x) \Vert \) where \(\rho '_d(x)\) is a diffuse albedo value of the compressed diffuse reflectance function \(R_d\).
As shown in Table 2, a smaller value of K results in smaller BSSRDF data, whereas fewer relative errors occur for a larger value of K, as can be seen in Fig. 5. To provide a more fair comparison between the cases of \(K = 1\) and \(K = 2\), we also provide the results for \(K = 1\), which has data sizes that are matched to those of \(K = 2\) by changing the value of M. By comparing these results in Fig. 5, it is determined that both the reconstructed appearance and relative errors of \(K=2\) are superior to those of \(K = 1\), even if the size of the resulting data is equal. Specifically, anisotropic light scattering is appropriately reproduced by the compressed BSSRDF of \(K = 2\), as shown in Fig. 5d. The improved reproducibility of \(K=2\) compared to \(K=1\) is considered because the number of base scattering profiles used to represent a value of \(R_d(x_i, x_o)\) is larger for \(K = 2\) compared to \(K = 1\). Although only two base scattering profiles are used in the case of \(K = 1\), four base scattering profiles are used in the case of \(K = 2\). By comparing the relative errors of \(K = 1, M = 128\) and \(K = 1, M = 1221\), it was observed that the error and scattering heterogeneity are not improved by increasing the value of M. According to these observations, \(K=2\) is more preferable than \(K=1\) for the efficient reproduction of the input BSSRDF with a specific amount of data.
Rendering performance of the multiresolution splatting algorithm [20] used with BSSRDFs of the 2BMPP model
Following the preceding discussion, the case of \(K = 2\), i.e., the 2BMPP model, is primarily considered in the following experiments.
4.2 Realtime rendering
Hardware configurations of the tablet and mobile used in the experiment
HTC Nexus 9 (tablet)  Sony F5321 (mobile)  

CPU  Dual core 2.3 GHz  Hexa core 1.8 GHz 
GPU  NVIDIA Kepler DX1  Qualcomm Adreno 510 
RAM  1.8 GB  2.8 GB 
Comparison of the average relative errors of compressed BSSRDFs among the proposed method, the previous method by Song et al. [5], and the LSPclusters
Material  Art. stone  Chessboard  Onyx  Blue wax  Yellow wax  Jade 

Avg. relative error  
Song et al. [5]  0.0223  0.0376  0.0413  0.0237  0.0222  0.0610 
LSPclusters  0.0429  0.0438  0.0435  0.0402  0.0619  0.0807 
1BMPP  0.0303  0.0425  0.0434  0.0441  0.0351  0.0728 
2BMPP  0.0372  0.0438  0.0428  0.0276  0.0319  0.0753 
3BMPP  0.0413  0.0452  0.0430  0.0273  0.0307  0.0789 
5BMPP  0.0491  0.0444  0.0427  0.0345  0.0370  0.0773 
Comparison of PSNR and SSIM among our compression method, the previous study by Song et al. [5], and the LSPclusters. Since the PSNR and SSIM are metrics for comparing image qualities, we compared the appearance of the reconstructed material under uniform illumination as in Fig. 5. In addition, the number of base materials K used for the results of the LSPclusters and 1BMPP are controlled such that the data sizes are the same as those of 2BMPP
Material  Art. stone  Chessboard  Onyx  Blue wax  Yellow wax  Jade 

PSNR (dB)  
Song et al. [5]  39.7  28.8  18.8  40.5  36.8  26.6 
LSPclusters  25.3  25.2  18.0  21.3  21.4  19.6 
1BMPP  31.3  26.4  18.0  20.7  28.8  20.5 
2BMPP  30.7  26.5  18.6  27.3  31.9  20.9 
3BMPP  26.7  24.8  18.5  27.1  32.9  20.4 
5BMPP  24.6  26.8  18.3  25.3  31.7  20.9 
SSIM  
Song et al. [5]  0.998  0.960  0.888  0.999  0.995  0.977 
LSPclusters  0.875  0.855  0.835  0.949  0.949  0.852 
1BMPP  0.970  0.940  0.853  0.843  0.942  0.821 
2BMPP  0.937  0.905  0.870  0.964  0.962  0.823 
3BMPP  0.901  0.873  0.870  0.974  0.962  0.793 
5BMPP  0.854  0.901  0.873  0.959  0.952  0.811 
4.3 Experiments
Figure 9 compares the appearances of light scattering among the original data, previous method [5], the LSPclusters, and the proposed method. We used \(M = 128\) base materials for the proposed method and chose the number of clusters for the LSPclusters such that the resulting data size was the same as that of the proposed method using \((K, M) = (2, 128)\). In this figure, the reconstructed appearances under uniform, circleshaped, and lineshaped illuminations are shown from left to right, and the relative errors of the diffuse albedo are shown in the rightmost column. By comparing the reconstructed appearances of the 2BPMM model, previous method, and original data, it was determined that all their appearances are nearly identical. According to the relative errors, the 2BMPP model performs comparably to the previous method [5] even though the data size of the proposed method is significantly smaller. These results demonstrate that the proposed method with two base materials effectively reduces redundancy in Song et al.’s previous method. Comparing the proposed method with the LSPclusters, it was determined that the reconstructed appearances and relative errors are almost the same. We further performed a quantitative comparison of the proposed method with the LSPclusters as discussed in the following paragraphs.
The values in Tables 5 and 6 are computed by comparing the reference BSSRDF data and the compressed data obtained using the proposed method or the previous method. The PSNR and SSIM values are only slightly lower than those of the previous method even though the data size obtained using the proposed method is significantly smaller than those obtained by the previous method. In addition, compared to the results for the LSPclusters, the proposed method with 2BMPP outperforms the LSPclusters for all values except in the case of the SSIM for “Jade.” For the “Jade” example, the SSIM values of the proposed method are smaller than that of the previous method. The resulting base material assignments can sometimes converge insufficiently for such precise and complex mixtures, even for an increase in the number of base materials K. The development of a more efficient optimization for material assignment will be addressed in future work.
Figure 10 shows the reproduced appearances and relative errors of the diffuse albedo for different numbers of scattering profiles. As can be seen in this figure, the reproduced appearances obtained with 8 and 32 base materials are sufficiently comparable with those obtained using the original data. As the number of profiles increases, the details of the reproduced appearance become clearer. For example, the appearance of the small white circles in the upper left of “Art. stone” becomes close to that of the original data, and the vertical lines of the “Onyx” become more apparent. According to these results, users can flexibly select the number of base materials depending on their size requirements for BSSRDF data. Even when \(M = 128\) is selected for high visual quality, the resulting data size is still significantly smaller than that of the previous methods as shown in Table 2.
4.4 Discussion
4.4.1 Target object shapes
The compression results shown in this paper are all for the BSSRDFs on flat objects, even though such flat shapes are not common in realworld materials. On the other hand, the proposed compression method is independent of the objects shape. Therefore, it can be extended to the case of nonflat objects as far as the distance between two surface positions can be calculated. Such BSSRDFs for general shapes are measured, for example, by the method of Goesele et al. [1], and our compression method can be immediately applied to their measurements. When we use the measured BSSRDFs in realtime rendering, the BSSRDFs measured for flat objects can be applied to nonflat shapes as shown in Fig. 6 based on the assumption that the target shapes are locally flat and the materials are optically thick.
4.4.2 Computational time for optimization
To obtain compressed BSSRDFs of KBMPP models with \(K \ge 3\), the current implementation is timeconsuming as shown in Table 2 because the optimization problem for \(K \ge 3\) needs to be solved using the slowconvergence augmented Lagrangian method. The computational bottleneck in the optimization is the evaluation of the least squares cost function in Eq. 5 and its derivative. However, evaluations of these cost function and its derivative can be accelerated with GPUs. Therefore, the problem of the computational time can be reasonably addressed. In addition, the 2BMPP models, which can be yielded in a significantly reduced computational time, are practically sufficient in most cases as shown by the experimental results in the previous subsections.
4.4.3 Random versus clusterbased initialization
As described in Sect. 3, the parameters of the compressed BSSRDF are initialized with random numbers before optimization. On the other hand, these parameters can be initialized with an input BSSRDF data. For example, the base scattering profiles can be initialized by clustering the local scattering profiles of the input pixels. However, it was determined in the preliminary experiments that such initialization by clustering can easily be stuck in a local minimum of the cost function in Eq. 5. This is because clusterbased initialization tends to inhibit updating of base material assignment by the preceding optimization steps. As a result, this inappropriate initialization can cause approximation errors at material boundaries. This is also the reason why the probabilistic update strategy was used in the PatchMatchbased material assignment. If material assignment is always updated as in the original PatchMatch algorithm, the solution can be stuck in a local minimum of the cost function.
4.5 Limitations
Except for the slight increase in the approximation errors, the proposed method with 2BMPP does not have any critical limitations. One possible problem is the many optimization parameters in the proposed optimization method. As far as the proposed method was examined with the parameters indicated in the paper, the optimization problems were robustly solved for the 2BMPP model.
On the other hand, if the user specifically needs to approximate a measured BSSRDF using more than two base materials, the proposed method has several limitations as we discussed in the previous paragraphs. First of all, the computation time for the case of \(K \ge 3\) is impractically long as shown in Table 2. This long computation time is due to the need for many iterations to allow the modeling parameters to converge. In addition, as indicated in Tables 5 and 6, approximation results given with \(K \ge 3\) can be worse than the case of \(K = 2\). A possible reason for this problem is the slow convergence of the augmented Lagrangian method, which can complicate the appropriate assignment of the base materials. Another possible reason is the large search space for the combinations of the base materials in the cases of \(K \ge 3\). Suppose that the typical test case of \(M = 128\) and a BSSRDF is defined on \(256 \times 256 = 65{,}536\) pixel area. When \(K = 2\) is used, the number of possible base material combinations is \({}_{128} C_{2} = 8128 < 65{,}536\). Meanwhile, it increases to \({}_{128} C_{3} = 341{,}376 > 65{,}536\) when \(K = 3\) is used. Since the proposed combinatorial optimization does not change the combinations assigned by the random initialization, the increase in the base material combinations can complicate the appropriate assignment of the appropriate combinations.
In conclusion, we recommend the use of the 2BMPP model in most practical cases given its numerical instability, relatively low computation time, and good approximation for many heterogeneous translucent materials.
5 Conclusion
In this paper, we proposed a memoryefficient BSSRDF model for the measurement of heterogeneous translucent materials. The high compression ratio of the proposed method is typically preferred for memorylimited devices such as video game consoles and mobile phones. The proposed method compresses physically measured BSSRDFs into significantly smaller data sizes compared to the stateoftheart compression method by Song et al. [5]. The compression is based on the observation that most translucent materials are comprised of only a limited number of base materials. The compression method represents the local scattering profile of each pixel with a linear combination of the scattering profiles of base materials in the log scale. In the proposed method, we assign multiple indices of the base materials and mixture ratios to each pixel. However, optimizing these parameters is a complicated problem in which constrained nonlinear optimization and NPhard combinatorial optimization must be simultaneously solved. To solve this complex problem, we proposed an alternating optimization method using the LBFGSB method [28] or an augmented Lagrangian method [27] for the constrained nonlinear optimization and a randomized approach inspired by PatchMatch [6] for the combinatorial optimization. The resulting compressed data are approximately 30 times smaller than the data obtained using the stateoftheart method. In addition, the visual quality of the rendered images compared to the compressed BSSRDFs is nearly identical to that obtained with the previous method. Several experiments have confirmed that the small data size obtained using the proposed method is preferable in practice for memorylimited devices. Apart from the direct utilization of the proposed compression method in memorylimited devices, its high compression ratio is expected to be more important when highresolution BSSRDFs are used in practical applications.
Footnotes
 1.
Kotlin is a modern multiplatform programming language. It primarily focuses on accelerating the development of Android applications, and its compiled binary is executed on Java VM.
Notes
Acknowledgements
The authors would like to thank Pieter Peers, Xin Tong, and Yue Dong for providing their heterogeneous BSSRDF datasets, which made this study possible. The authors would also like to thank the anonymous reviewers for their constructive comments and helpful suggestions which assisted in significantly improving the quality of this paper.
Funding
Tatsuya Yatagawa was supported by a Research Fellowship for Young Researchers from Japan’s Society for the Promotion of Science (GrantinAid for JSPS Fellows JP16J02280). Hideki Todo was supported by JSPS KAKENHI Grant No. JP15H05924. Yasushi Yamaguchi was supported by JSPS KAKENHI Grant No. JP16H02818. Shigeo Morishima was supported by JST ACCEL Grant No. JPMJAC1602, and a GrantinAid from the Waseda Institute of Advanced Science and Engineering.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Supplementary material
Supplementary material 2 (mp4 157701 KB)
References
 1.Goesele, M., Lensch, H.P.A., Lang, J., Fuchs, C., Seidel, H.P.: Disco: acquisition of translucent objects. ACM Trans. Graph. 23(3), 835 (2004). https://doi.org/10.1145/1186562.1015807 CrossRefGoogle Scholar
 2.Peers, P., vom Berge, K., Matusik, W., Ramamoorthi, R., Lawrence, J., Rusinkiewicz, S., Dutré, P.: A compact factored representation of heterogeneous subsurface scattering. ACM Trans. Graph. 25(3), 746 (2006). https://doi.org/10.1145/1179352.1141950 CrossRefGoogle Scholar
 3.Wang, J., Zhao, S., Tong, X., Lin, S., Lin, Z., Dong, Y., Guo, B., Shum, H.Y.: Modeling and rendering of heterogeneous translucent materials using the diffusion equation. ACM Trans. Graph. 27(1), 9:1 (2008). https://doi.org/10.1145/1330511.1330520 Google Scholar
 4.Nicodemus, F.E., Richmond, J.C., Hsia, J.J., Ginsberg, I.W., Limperis, T.: Geometrical Considerations and Nomenclature for Reflectance. National Bureau of Standards, Gaithersburg (1977)CrossRefGoogle Scholar
 5.Song, Y., Tong, X., Pellacini, F., Peers, P.: Subedit: a representation for editing measured heterogeneous subsurface scattering. ACM Trans. Graph. 28(3), 31–1 (2009). https://doi.org/10.1145/1576246.1531337 CrossRefGoogle Scholar
 6.Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 24:1 (2009). https://doi.org/10.1145/1531326.1531330 CrossRefGoogle Scholar
 7.Chandrasekhar, S.: Radiative Transfer. Oxford University Press, Oxford (1960)zbMATHGoogle Scholar
 8.Jensen, H.W., Marschner, S.R., Levoy, M., Hanrahan, P.: A practical model for subsurface light transport. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’01), pp. 511–518 (2001). https://doi.org/10.1145/383259.383319
 9.Donner, C., Jensen, H.W.: Light diffusion in multilayered translucent materials. ACM Trans. Graph. 24(3), 1032 (2005). https://doi.org/10.1145/1073204.1073308 CrossRefGoogle Scholar
 10.d’Eon, E., Irving, G.: A quantizeddiffusion model for rendering translucent materials. ACM Trans. Graph. 30(4), 56 (2011). https://doi.org/10.1145/2010324.1964951 Google Scholar
 11.Frisvad, J.R., Hachisuka, T., Kjeldsen, T.K.: Directional dipole model for subsurface scattering. ACM Trans. Graph. 34(1), 5:1 (2014). https://doi.org/10.1145/2682629 CrossRefzbMATHGoogle Scholar
 12.Frederickx, R., Dutré, P.: A forward scattering dipole model from a functional integral approximation. ACM Trans. Graph. 36(4), 109:1 (2017). https://doi.org/10.1145/3072959.3073681 CrossRefGoogle Scholar
 13.Song, Y, Wang, W: AsiaPacific Signal and Information Processing Association Annual Summit and Conference, pp. 1–7 (2013). https://doi.org/10.1109/APSIPA.2013.6694333
 14.Stam, J.: Multiple Scattering as a Diffusion Process, pp. 41–50. Springer, Vienna (1995). https://doi.org/10.1007/9783709194300_5 Google Scholar
 15.Kaplanyan, A., Dachsbacher, C.: Cascaded light propagation volumes for realtime indirect illumination. In: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 99–107 (2010). https://doi.org/10.1145/1730804.1730821
 16.Børlum, J., Christensen, B.B., Kjeldsen, T.K., Mikkelsen, P.T., Noe, K.Ø., Rimestad, J., Mosegaard, J.: In: Proceedings of the ACM SIGGRAPH Symposium on High Performance Graphics (ACM, 2011), HPG ’11, pp. 7–14. https://doi.org/10.1145/2018323.2018325
 17.Koa, M.D., Johan, H.: ESLPV: enhanced subsurface light propagation volumes. Vis. Comput. 30(6), 821 (2014). https://doi.org/10.1007/s0037101409523 CrossRefGoogle Scholar
 18.Lensch, H.P.A., Goesele, M., Bekaert, P., Kautz, J., Magnor, M.A., Lang, J., Seidel, H.P.: Interactive rendering of translucent objects. In: 10th Pacific Conference on Computer Graphics and Applications, pp. 214–224 (2002). https://doi.org/10.1109/PCCGA.2002.1167862
 19.Shah, M.A., Konttinen, J., Pattanaik, S.: Imagespace subsurface scattering for interactive rendering of deformable translucent objects. IEEE Comput. Graph. Appl. 29(1), 66 (2009). https://doi.org/10.1109/MCG.2009.11 CrossRefGoogle Scholar
 20.Chen, G., Peers, P., Zhang, J., Tong, X.: Realtime rendering of deformable heterogeneous translucent objects using multiresolution splatting. Vis. Comput. 28(6), 701 (2012). https://doi.org/10.1007/s0037101207041 CrossRefGoogle Scholar
 21.Jimenez, J., Gutierrez, D.: Screenspace subsurface scattering. In: GPU Pro: Advanced Rendering Techniques, pp. 335–351 (2010). https://doi.org/10.1201/b1064826
 22.Jimenez, J., Zsolnai, K., Jarabo, A., Freude, C., Auzinger, T., Wu, X.C., der Pahlen, J., Wimmer, M., Gutierrez, D.: Separable subsurface scattering. Comput. Graph. Forum 34(6), 188 (2015). https://doi.org/10.1111/cgf.12529 CrossRefGoogle Scholar
 23.Dachsbacher, C., Stamminger, M.: Translucent shadow maps. In: Proceedings of the 14th Eurographics Workshop on Rendering, pp. 197–201 (2003). https://doi.org/10.2312/EGWR/EGWR03/197201
 24.Mertens, T., Kautz, J., Bekaert, P., Seidelz, H.P., Van Reeth, F.: Interactive rendering of translucent deformable objects. In: Proceedings of the 14th Eurographics Workshop on Rendering, pp. 130–140 (2003). https://doi.org/10.2312/EGWR/EGWR03/130140
 25.Dal Corso, A., Frisvad, J.R., Mosegaard, J., Bærentzen, J.A.: Interactive directional subsurface scattering and transport of emergent light. Vis. Comput. 33(3), 371 (2017). https://doi.org/10.1007/s0037101612072 CrossRefGoogle Scholar
 26.Jensen, H.W., Buhler, J.: A rapid hierarchical rendering technique for translucent materials. ACM Trans. Graph. 21(3), 576 (2002). https://doi.org/10.1145/566654.566619 CrossRefGoogle Scholar
 27.Conn, A.R., Gould, N.I., Toint, P.: A globally convergent augmented lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28(2), 545 (1991). https://doi.org/10.1137/0728030 MathSciNetCrossRefzbMATHGoogle Scholar
 28.Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190 (1995). https://doi.org/10.1137/0916069 MathSciNetCrossRefzbMATHGoogle Scholar
 29.Kotlin Programming Language. https://kotlinlang.org/. Accessed 20 Dec 2017
 30.Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600 (2004). https://doi.org/10.1109/TIP.2003.819861 CrossRefGoogle Scholar
Copyright information
OpenAccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.