Keywords

1 Introduction

Skin is one of the largest and most complex organs found in the human body. Its appearance is caused by several factors that can be clustered into two main ones: internal (endogenous) and external (exogenous). These factors determine the final skin structure and composition. The quantity of information provided by skin properties is enough to describe an individual and distinguish him from others. For example, it is possible to easily recognize the identities, sex and age range, lifestyle or health condition by observing people’s faces. Human traits are perceived thanks to vision, which completely depends on the presence of light.

The electromagnetic radiation found between 350 nm and 750 nm is what we acknowledge as the visible light spectrum. Visual perception is based on reflected light radiation. Common representation techniques for imaging make use of color-spaces or colorimetry. These approaches portray some skin qualities like its tonality but are highly dependent on the illumination level used. For example, in RGB color space the intensity for each color channel is defined as the integral of all energy sensed over the pixel.

State of the art techniques on subcutaneous prediction and evaluation endorse using reflectance based analysis. Several materials present unique traits in the spectral distribution of their reflectance. For skin, the analysis of its perception is known as optical biopsy and has applications in many areas of expertise such as demographics, medicine or cosmetics. Moreover, the acquisition of reflectance information is performed using non-invasive maneuvers. It is identified as the ratio between reflected and incident light energy thus implying that the information obtained is normalized.

1.1 Skin Anatomy

Skin complexity comes from its multiple layers, properties and particle presence. Although composition, size and structure is unique to each person some share range similarities [1]. The layers that are responsible for most of the appearance perceived from skin are those closest to the surface: epidermis and dermis. These two layers possess a combined thickness ranged between 0.5 mm and 4 mm with the dermis being approximately 24 times thicker. Epidermis is the outer-most layer which reveals the aspect and health condition of skin because it carries the main aesthetics of the body. Water presence in the epidermis is estimated around 10–20% and determines how bright or dry it is perceived. Epidermal thickness along with the presence of melanin cause the final aspect perceived by the skin through this layer.

The dermis layer is connected to the upper epidermal layer and consists of several cells, fiber and substances distributed into 2 sub-layers. These fibers belong to collagen and elastin structures. Up to 70% of dry skin weight comes from collagen distributed into thin and thick structures forming a complex rigid network that protect the blood vessels inside them. Hemoglobin is transported with the blood torrent and is the main responsible for blood vessel reddish tonality. Its concentration is closely related to blood circulation.

1.2 Skin Visual Perception

Because skin is a semi-translucent material, light trespasses its surface and interacts with the medium. Higher wavelengths reach deeper than the dermis layer. Pigments and structures are responsible for two different phenomena that define the reflectance spectra: absorption and diffraction [2]. Absorption corresponds to the loss of energy and can be mainly attributed to the presence of pigments and dried tissue in skin. On the other hand, diffraction relates to the trajectory deviation of incident light which causes that some wavelengths stay longer in the medium before exiting. This phenomenon is associated to the presence of collagen structures. The perceived skin aspect is mainly caused by the previous phenomena at the epidermis and dermis layers. Thinner skins present lighter tones while darker tonalities imply higher thickness and pigment presence. Due to the variations in skin anatomy among individuals we propose the usage of reflectance image analysis techniques for demographic classification.

1.3 Skin Related Demographics

As previously introduced, thickness and pigmentation are responsible for how skin is perceived. It is also one of the most demographic-related features as it varies from location to location, age range, ethnicity and gender. For example, female population usually present thinner skin than males (Poulsen et al. [15]) or younger people possess higher skin thickness than the elderly. In the case of an adult human it corresponds to almost 3.5 times thicker than infant skin tissue, but children do not exhibit similar skin thickness before reaching 5 years of age. It is known that epidermal thickness reaches it maximum at the age of 20 while for the dermis layer this event occurs at 30. Thicker layers mean a darker perceived skin tone.

Pigmentation presence is closely related to ethnic traits. However, the influence of age and solar radiation also intervene in the concentration of pigments in skin. Melanin is created as a response to the increase of subcutaneous metabolism. It is clear that sun exposure increases with age and thus the amount of skin damage and the growth in melanin presence. Males also present a higher level of melanin than women which is related to lighter skin tones across female population [8]. The elasticity of skin is related to the status of its collagen structures and as mentioned before, sun exposure and age wages on skin health. These are only some identifiable traits regarding demographics and the most defining properties of skin.

2 Related Work

Several studies indicate the presence of differences in pigmentation and skin structure between ethnicities, gender and age ranges [15]. To expand on these facts, we present the state of the art in reflectance image analysis to evaluate the feasibility of demographic classification using these techniques.

2.1 Skin Classification Based on Skin Aging and Light Exposure

Fitzpatrick’s skin classification [7] is widely used in clinic and cosmetics as it establishes appearance as a response to possible effects of light exposure: sunburn and tanning capabilities of skin. Thus, it analyzes how different skin types react to light rather than associating ethnic traits to skin tonalities. Fitzpatrick identifies a total of 6 different skin types non directly related to color. More specifically, it considers the damage suffered on skin under UV exposure and melanin levels produced due to it. Fitzpatrick skin descriptions are provided in Table 1.

Table 1. Fitzpatrick skin type classification
Full size table

Under analysis of each Fitzpatrick’s group, we identified a relation between types I to III and caucasian traits while IV and V corresponded to brownish tonalities. Finally, type VI identifies darker skin tones. However there is not a direct relation between the classification types and skin aging which can come from natural or photo-aging from UV exposure. Photo-aging causes dryness, hyper-pigmentation and yellowing of the skin.

2.2 Light-Skin Interaction Prediction: BSSRDF

To perform the association between light propagation optics and skin biological properties a light-skin interaction model is required. Most models are based on the Radiative Transfer Equation (RTE) theory by Chandrasekhar [3] and Ishimaru [11] which describes the light radiation variation as the emitted, reflected and transferred light at an arbitrary point in the medium. For skin, these are defined by the absorption and scattering phenomena.

The Bidirectional Subsurface Scattering Reflectance Distribution Function (BSSRDF) is one model to describe the interaction. It determines the outgoing location of a light beam that entered the medium, which can be different from the entry point. BSSRDF [5] takes into consideration the absorption and scattering events in propagation, and is defined by the ratio of outgoing to the incoming radiance:

$$\begin{aligned} S(x_i,\overrightarrow{\omega }_i,x_o,\overrightarrow{\omega }_o) = \frac{S(x_o,\overrightarrow{\omega }_o)}{S(x_i,\overrightarrow{\omega }_i)} \end{aligned}$$
(1)

The BSSRDF describes the reflectance of the material as follows:

$$\begin{aligned} S(x_i,\overrightarrow{\omega }_i,x_o,\overrightarrow{\omega }_o) = \frac{1}{\pi }F_t(\eta ,\overrightarrow{\omega }_i)R(||x_i-x_o||)F_t(\eta ,\overrightarrow{\omega }_o) \end{aligned}$$
(2)

where \(F_t\) is the Fresnel transmittance term and \(\textit{R}\) is the reflectance of the material. On the other hand, the BRDF is a particularization of the BSSRDF where the light beam exits at the same point of entry. The BSSRDF can also describe the absorption and scattering phenomena in translucent materials.

2.3 State of the Art Skin Models

Several models can be applied to simulate the light-skin interaction that produces the reflectance signal depending on the accuracy and level of demand desired. Most models take into account the RTE to describe the different light interactions inside the medium.

The Monte Carlo model [16] simulates the physical phenomena in probabilistic and statistical manners and the propagation an interaction processes are described by the RTE. However the solution to the RTE is rather accurate. Accuracy increases when arbitrarily incrementing the number of process simulations. The result of the model provides statistical information regarding the absorption, reflection and transmission processes of the material. Figure 1 presents the Monte Carlo simulation steps to obtain these results.

Fig. 1.
figure 1

Monte Carlo model for reconstructing light-skin interaction

Full size image

On the other hand, the multipole model reduces the computation time required for the Monte Carlo model. Proposed by Jensen et al. [10] it simulates subsurface light transport in a high scattering medium like skin. The RTE is solved through approximating how light diffuses for each point in the medium. This presents a high difficulty level and can only be achieved by considering an isotropic medium. The last assumption is valid because skin is an anisotropic medium with strong scattering properties and thus light distribution becomes isotropic after multiple scattering events. If the medium were infinite, the solution to the equation would be the same as for Green’s function. However this is not the case with skin and thus an expansion of the model is required to finally compute the reflectance in a multi-layered material.

Finally, the Kubelka-Munk model [12] presents a good computational speed and simplicity for skin analysis. It locally models light transport with a simple relation between optical medium properties and its reflectance. Given a medium slab with infinitely wide and length, absorption and scattering coefficients \(\sigma _a\) and \(\sigma _s\) respectively, the incident flux is modeled as 2 flux with opposite directions after passing a small distance. After traversing a dx thickness, each flux is partially absorbed and scattered. Downward flux I decreases due to absorption and scattering and increases because of the scattering of J as written in Eq. 3.

$$\begin{aligned} \left\{ \begin{array}{c} dI=-(\sigma _a+\sigma _s)Idx+\sigma _sJdx\\ -dJ=-(\sigma _a+\sigma _s)Jdx+\sigma _sIdx\\ \end{array} \right. \end{aligned}$$
(3)

3 Proposed Albedo Based Descriptor

Most demographic classification techniques are appearance based. Results present successful rates but values decrease for darker skin tonalities or bad illumination causing traits to be less perceivable. The usage of reflectance albedo provides color information of the skin while also overcoming the previous drawbacks. We present an RGB albedo descriptor for the depiction of skin properties based on the reflectance measured in each of its three color channels.

3.1 RGB Albedo Descriptor

The skin albedo quantifies the absorption and scattering ability dictated by the subcutaneous properties and demands the acquisition of the skin BSSRDF for that. Nevertheless, it is often approximated as the BDRF for simplicity and as a result, the skin albedo is estimated as the minimum ratio between BRDF \(f_r\) and Fresnel terms \(F_t\) as follows:

$$\begin{aligned} \alpha (x)= \underset{i}{min}\frac{\pi f_{r}(x,\overrightarrow{\omega }_o, \overrightarrow{\omega }_i)}{F_{t}(\eta ,\overrightarrow{\omega }_i)F_{t}(\eta ,\overrightarrow{\omega }_o)} \end{aligned}$$
(4)

where \(\overrightarrow{\omega }_i\) and \(\overrightarrow{\omega }_o\) are the incoming and outgoing light directions at surface point x, \(f_r\) is the surface BRDF, \(F_t(\eta ,\overrightarrow{\omega })\) is the Fresnel transmittance term with light direction \(\overrightarrow{\omega }\), \(\eta \) is the relative refractive index between skin and air and is often assumed to be constant throughout all facial skin. The acquisition of skin BRDF can be performed using digital cameras, as proposed in the works of Weirich et al. [17]. From the skin diffuse albedo we extract our skin descriptor.

Fig. 2.
figure 2

R, G and B separate albedo histograms, and resulting concatenation (Color figure online)

Full size image

For each RGB channel, the albedo is divided into N equally distributed intervals between 0 and 0.4 because the skin RGB albedo generally stands in this range. After that, we compute the albedo distribution histogram as:

$$\begin{aligned} H(i) = \frac{n_i}{s\sum _{j=1}^{N}n_j} \end{aligned}$$
(5)

where H(i) is the value for the i-th bin, \(n_i\) is the quantity of albedo that corresponds in the \(i-th\) interval, and s is the used step size between adjacent albedo intervals. The histogram satisfies the following properties:

$$\begin{aligned} \left\{ \begin{array}{l} H(i) >= 0, i= 1,2,\ldots , N \\ \sum _{i=0}^{N}\, H(i)s= 1 \\ \end{array} \right. \end{aligned}$$
(6)

Finally, by concatenating the R, G and B albedo histograms in order we obtain the proposed albedo-based descriptor as shown in Fig. 2.

4 Demographic Classification Results and Analysis

To evaluate the albedo-based descriptor performance, the MERL/ETH [18] database was used. It contains 156 subjects aged between 13 and 74 years with labeled gender. The diffuse albedo is obtained from 10 different facial regions as listed in Table 2. This section is divided into 2 parts. First, the data was clustered using k-means and compared with biological results. Then, we compare the performance for age, gender and Fitzpatrick skin type classification.

Table 2. Regions in MERL/ETH database
Full size table

4.1 Gender, Skin Type and Age Clustering

The clustering process allows the establishment of relations between skin properties (age, gender and skin type) and RGB albedo information. We used the k-means algorithm for clustering subject albedo data. Initial cluster centers were randomly selected and the number of clusters was specified. Execution of the algorithm was performed enough times and final cluster center results were those that provided the lowest sum of point-to-center distances.

Table 3. Gender clustering results
Full size table

We obtained 3 different clusters for gender classification, as showed in Table 3. This table shows the value of the RGB albedo values related to each cluster center and the following conclusions were extracted. First, the proportion of women in cluster 1 is very low. Thus, women present higher reflectance meaning that their skin is thiner than men’s. This is also described by the experiments in Giacomini et al. [6]. The small segment of women in cluster 1 present darker skin tones.

In regards to the skin type clustering, we defined 5 groups according to the work by Fitzpatrick. This is due to the combination of groups I and II from Table 1 because subjects from I were scarce for clustering. Initially, lighter tonalities were associated to groups I to IV, brownish for V and dark for group VI. Groups I to III can manifest sunburns, while the IV to VI present difficulties to do that. The skin tanning level is perceivable in groups III to VI. After processing the algorithm, two clusters remain. The first one is composed by types I to III, while the second is conformed by types V and VI, which showed similar albedo properties. Group IV fell as an intermediate point in between, as displayed in Table 4. Results present high similarity to the work of Fitzpatrick on skin type categorization.

Table 4. Gender clustering results
Full size table
Table 5. Age range clustering results for type III males
Full size table
Table 6. Age range clustering results for type IV males
Full size table

Finally, for age clustering four different groups were initially defined: under 25, 25 to 30, 30 to 40 and above 40. Over this initial assumption, two clusters appear for each of the previously defined ranges. Most concise data for the obtained clusters were related to masculine skin type III and IV respectively. Their data was analyzed to extract age effect. First, Table 5 shows the data related to type III skin male subjects, while Table 6 presents those for type IV.

From this information we can denote the standard RGB reflectance (cluster centers) increase in variation with age which ultimately translates as that color homogeneity of classes decreases with age.

4.2 Gender, Skin Type and Age Classification

For the classification tasks, RBF-kernel Support Machine Vector (SVM) of age, gender and skin type was performed. Subjects from each class were randomly chosen for cross validation, training and testing also making up for over-sized or undersized classes. The cross validation consisted in a 10-fold approach partitioning data so that 7 sets were dedicated to training, 2 for testing and the one remaining was used for fusion training of the classifier. A total 10 rounds of this procedure were performed and the best result was kept. We analyzed data of the 6 most data-populated facial regions (1, 2, 6, 7, 8 and 9) and results obtained were as displayed in Table 7.

Table 7. Classification results using data from different regions
Full size table

Region 2 showed the better classification results while region 8 presented the worst performance rates overall under the proposed approach. Despite the improvable results in age, gender classification achieved good performance, specially for region 2. We used the class information provided by the SVM for late fusion of the data from the 5 best performing face regions. Each region is given a weight from the information provided by the SVM. Region fusion results are shown in Table 8.

Table 8. Classification results fusing data from different regions
Full size table

Fusion increases gender classification results by 3.44%. However, there is a significant increase in Fitzpatrick skin type (13.75%) and age range classification (17.82%), which resulted in overall categorization rates above 90% for all categories.

Finally, we analyzed the correlation between weights and average skin thickness, as described in Ha et al. [9]. For gender and age classifications, the weight increased along with skin thickness. This fact is supported by the works of Ortonne [13], where it is cited that sunlight exposure stimulates hyper-pigmentation emergence. Males showed thicker skins which contain more melanin causing darker tonalities. However, Fitzpatrick skin type classification showed weight decrease when thickness increased. This was thought to be related to Fitzpatrick’s association of higher pigmentation being responsible of low tanning capability of skin.

5 Conclusion

Typical imaging methods are highly dependent of illumination levels. Albedo based techniques provide a more stable and accurate solution to extract information. The proposed approach was based on the usage of skin reflectance RGB albedo for demographic classification under gender, age range and skin type. The proposed RGB albedo descriptor is produced concatenating the reflectance information measured over each one of the RGB channels. As a result, we obtain skin color normalized information not conditioned by light presence levels.

An initial clustering process was performed with albedo data to confirm skin properties hypothesis regarding thickness differences between gender, increase of pigmentation with age and for darker toned ethnicities. As for classification capabilities, different facial region albedo samples were used. Best results when individually analyzing the regional albedos provided close to 90% for gender, age range and Fitzpatrick skin type. In addition to these results, a fusion algorithm using each of the regional albedo was use to evaluate overall face performance for classification. Final results increased total classification capabilities of 96.88%, 95% and 90.63% for age, gender and skin type respectively. The previous numbers endorse our proposed approach for skin-based classification.

After evaluating the followed methodology and obtained results, we propose the inclusion of albedo information from additional color channels or spectral bands to increase performance in skin analysis and demographic classification.