Nothing Special   »   [go: up one dir, main page]
Next Article in Journal
Paradox Elimination in Dempster–Shafer Combination Rule with Novel Entropy Function: Application in Decision-Level Multi-Sensor Fusion
Previous Article in Journal
Fault Diagnosis of a Rotor and Ball-Bearing System Using DWT Integrated with SVM, GRNN, and Visual Dot Patterns
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 19
Issue 21
10.3390/s19214805
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis

by
Jacob Renzo Bauer
1,*,
Jean-Baptiste Thomas
1,
Jon Yngve Hardeberg
1 and
Rudolf M. Verdaasdonk
2
1
The Norwegian Colour and Visual Computing Laboratory, Norwegian University of Science and Technology (NTNU), 2815 Gjøvik, Norway
2
Biomedical Photonics and Imaging group, Faculty of Science and Technology, University of Twente, 7522NB Enschede, The Netherlands
*
Author to whom correspondence should be addressed.
Sensors 2019, 19(21), 4805; https://doi.org/10.3390/s19214805
Submission received: 10 September 2019 / Revised: 31 October 2019 / Accepted: 1 November 2019 / Published: 5 November 2019
(This article belongs to the Section Biomedical Sensors)
Browse Figures
Figure 1
<p>Framework, including training set, testing set, sensor sensitivities, reconstructed spectra and the evaluation according to RMSE, <math display="inline"><semantics> <mrow> <mo>Δ</mo> <msub> <mi>E</mi> <mn>00</mn> </msub> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mo>Δ</mo> <mi>O</mi> <mi>x</mi> <mi>y</mi> </mrow> </semantics></math>.</p> "> Figure 2
<p>Measured Munsell relfectances [<a href="#B40-sensors-19-04805" class="html-bibr">40</a>] (Munsell), measured skin reflectances [<a href="#B42-sensors-19-04805" class="html-bibr">42</a>] (SkinRef), simulated skin reflectances (SkinSim). Three reflectances highlighted for visibility in each set.</p> "> Figure 3
<p>Sensor sensitivities, one RGB camera [<a href="#B60-sensors-19-04805" class="html-bibr">60</a>], a prototypical implementation by Thomas et al. [<a href="#B3-sensors-19-04805" class="html-bibr">3</a>] (<span class="html-italic">France1</span>) and commercially available Silios [<a href="#B6-sensors-19-04805" class="html-bibr">6</a>] (Silios) (all <b>left</b>) and simulated GSB (GRGB, GFrance1 and GSilios) versions (all <b>right</b>).</p> "> Figure 4
<p>Sensor sensitivities, Ximea xispec [<a href="#B4-sensors-19-04805" class="html-bibr">4</a>] (Ximea and CorXim) (all <b>left</b>) and simulated GSB (GCorXim and GXimea) versions (all <b>right</b>).</p> "> Figure 5
<p>Dimensionality analysis of all sets combined (<b>B</b>) skin simulation (blue), skin reflectance (green) and Munsell reflectances (red). Colored markings for maximum PCA1, minimum PCA1, maximum PCA2, minimum PCA2, for skin simulation and skin reflectance, respectively. Color patch recreation (under D65 light source) of the extreme spectra for the skin simulation (<b>A</b>) and skin reflectance database (<b>C</b>) with minimum PCA 1 and PCA 2 and maximum PCA 1 and PCA 2. Plot of the maximum and minimum spectra for the skin reflectance database (<b>D</b>) and skin simulation database (<b>E</b>) according to PCA analysis.</p> "> Figure 6
<p>Resulting metrics of <math display="inline"><semantics> <mrow> <mo>Δ</mo> <msub> <mi>E</mi> <mn>00</mn> </msub> </mrow> </semantics></math> (D65, CIE 2<math display="inline"><semantics> <msup> <mrow/> <mo>∘</mo> </msup> </semantics></math> 1931) (<b>top</b>) and <math display="inline"><semantics> <mrow> <mi>R</mi> <mi>M</mi> <mi>S</mi> <mi>E</mi> </mrow> </semantics></math> calculated between reconstruction and training (<b>bottom</b>). All Sensors, Munsell set (<b>left</b>) and Skin Simulation set (<b>right</b>) as training including standard deviation of the resulting data. For all graphs, the filled “o” represents the original sensor and the “x” represents the GSB.</p> "> Figure 7
<p>Visualization of worst and best <math display="inline"><semantics> <mrow> <mo>Δ</mo> <msub> <mi>E</mi> <mn>00</mn> </msub> </mrow> </semantics></math> results for the uncorrected Ximea (Ximea top) and corrected Ximea (CorXim bottom), Munsell set (<b>left</b>) and Skin simulation (SkinSim) set (<b>right</b>) as training. Each graph includes GSB sensor results, ground-truth in solid lines and estimation with dashed lines.</p> "> Figure 8
<p>Resulting values for <math display="inline"><semantics> <mrow> <mo>Δ</mo> <mi mathvariant="bold-italic">Oxy</mi> </mrow> </semantics></math> metric calculated using six wavelength (500 nm, 520 nm, 540 nm, 560 nm, 580 nm, 600 nm) (<b>top</b>) and three wavelengths (480 nm, 560 nm, 600 nm) (<b>bottom</b>) for all Sensors. Munsell set (<b>left</b>) as training and Skin Simulation set (<b>right</b>) including standard deviation of the data.</p> ">
Versions Notes

Abstract

:
Comparing and selecting an adequate spectral filter array (SFA) camera is application-specific and usually requires extensive prior measurements. An evaluation framework for SFA cameras is proposed and three cameras are tested in the context of skin analysis. The proposed framework does not require application-specific measurements and spectral sensitivities together with the number of bands are the main focus. An optical model of skin is used to generate a specialized training set to improve spectral reconstruction. The quantitative comparison of the cameras is based on reconstruction of measured skin spectra, colorimetric accuracy, and oxygenation level estimation differences. Specific spectral sensitivity shapes influence the results directly and a 9-channel camera performed best regarding the spectral reconstruction metrics. Sensitivities at key wavelengths influence the performance of oxygenation level estimation the strongest. The proposed framework allows to compare spectral filter array cameras and can guide their application-specific development.

1. Introduction

Spectral filter array (SFA) cameras are a new single-shot spectral imaging technology [1], which is gaining popularity in different fields of research [2]. The light entering the camera is filtered with narrow spectral bandpass filters on each pixel or subpixel. Spatial decomposition of the spectral signal allows capturing of all spectral bands at the same instance.
Prototypes have been proposed in academia [3] and commercial models are now available including the XIMEA xiSpec camera [4,5] and Silios technologies SFA camera [6]. With increased adoption and commercial availability of SFA cameras, it is important to analyze parameters contributing to image quality parameters of these cameras and provide tools to guide further development for specific applications.
Image quality performance of cameras for close range imaging is a broad field of research [7,8,9] covering many different aspects including: spatial resolution [10,11,12], spectral or color accuracy [3,13,14], reproducibility, noise behavior [15], optical distortions and post-processing steps. The required accuracy of spectral reconstructions, number of channels and wavelength of interest are application dependent and should be evaluated in the context of specific applications. If SFAs combine accurate spectral reconstruction with real-time acquisition speed and ease of use, they could potentially be a powerful new imaging modality for the medical field. Digital imaging is already widely adopted for skin imaging, which could benefit from additional spectral information [16,17,18,19,20]. Small color variations in the skin can carry relevant information for physicians. There is a need for more reliable and quantitative methods to measure physiologic parameters of patients in non-contact. SFA cameras could combine non-contact monitoring of vital functions and diagnosis of diseased skin tissue in real time [21,22,23,24,25,26,27,28]. In particular, dynamic processes such as oxygenation would highly benefit from spectral and spatially resolved images in real time [27,29,30,31,32].
Previous work by Preece and Claridge [33] has investigated optimal filter sensitivities for a three-channel system for skin diagnosis. An extensive hardware focused analysis of spectral imagers for biomedical applications is provided by Gutiérrez-Gutiérez et al. [34]. The main focus of their work was the technical limitations including acquisition speed, efficiency, object plane curvature, spatial resolution, distortions, and noise. They emphasized an imaging system for biomedical applications should be selected after thorough testing of these parameters. A comprehensive emulation framework has been proposed by Saager et al. [35] giving an overview of the performance of different spectral imagers including a Xispec SFA camera and an RGB sensor a burn wound mouse model and photoaging experiment. High-resolution spectral measurements were performed using a spatial frequency domain spectroscopy (SFDS) system. In the computer graphic domain with Jimenez et al. [36] and Iglesias-Guitian et al. [37] described physically based skin appearance models to show color changes due to emotions or ageing. The same models can be used as to generate skin reflectance training sets.
The aim of this study is the development and testing of a framework for comparison of SFA cameras for spectral reconstruction, skin imaging, and oxygenation level estimation without prior patient measurements. A generated specialized training set is quantified for spectral reconstruction.
This framework could be considered prior to the hardware focused selection by Gutiérrez-Gutiérez et al. [34] and provides a simplified measurement free alternative to the method proposed by Saager et al. [35]. The framework could also be applied as a guide for the development of application-specific SFA cameras.
Three aims of study can be formulated as:
  • comparison framework of spectral filter array cameras for skin imaging and medical diagnosis
  • illustrate the impact of spectral reflectance reconstruction using a specialized training set for SFA camera applications in skin imaging.
  • recommendation of commercially available SFA cameras for monitoring of vital functions and diagnosis.

2. The Proposed Framework

The proposed framework has three main elements: (1) calculation of a spectral reconstruction matrix, (2) simulated sensor responses and (3) an evaluation block. It is shown in Figure 1 and follows the concepts of a spectral filter array processing pipeline proposed by Lapray et al. [38].
As a first part, a spectral reconstruction is performed to estimate the full spectra using the limited number of SFA bands providing a measure of the performance of the different cameras independent of applications. In addition, the estimated spectra are then analyzed regarding their accuracy for oxygenation level estimation being an example for a specific application. Three SFA cameras, one prototypical, two commercially available and an RGB camera are evaluated. The impact of gaussian spectral bands (GSB) is tested by simulating sensor sensitivities with gaussian shapes for each of the SFA cameras channels.
A set of (10,000) [39] skin reflectances is generated using a Monte Carlo skin model and compared to a Munsell reflectance patch database [40,41] for training the spectral reconstruction. A database of spectral measurements of skin reflectances (100 measurements) [42] is used for testing the spectral reflectance reconstruction. The spectral reconstruction accuracy is compared numerically using Root Mean Square Error (RMSE) and Δ E 00 color differences [43]. Differences in estimated oxygenation levels are numerically compared using a proposed metric. Spatial aspects are not considered in this study since the standard clinical measurement of oxygenation levels are usually averaged over a small area and the skin simulation is only considering homogeneous tissue over the simulated surface.

3. Prerequisites

For full spectral reconstruction simulated sensor responses are needed. The spectral reconstruction accuracy needs to be evaluated regarding spectral accuracy and in relation to specific applications. The framework could be applied to any channel-based spectral imager with known spectral sensitivities. For comparing specific spectral imagers, sensor sensitivities, training and test data and evaluation metrics must be chosen.

3.1. Spectral Imaging Model and Spectral Reconstruction

Spectral reconstruction is a useful estimation technique to estimate full spectra from a limited number of bands. The wavelengths of interest might also be unknown prior to the practical applications. It allows comparison of spectral cameras with different sensitivity peaks in a common space.
The spectral reconstruction is based on the inversion of a commonly known imaging model, which can be described with the equation:
P i = ∫ λ E ( λ ) R j ( λ ) Q i ( λ ) d λ
where P i is the channel response of the i t h channel of the sensor. E ( λ ) is the illumination spectral power distribution (SPD) per wavelength, R j ( λ ) is the spectral reflectance of sample j and Q i ( λ ) describes the spectral sensitivity of the i th channel of the sensor. Noise can be described as an additive constant to each channel.
Two simplification have been applied to the imaging model for this study. Noise per channel has not been considered and illumination has been assumed to be of equi-energy. Both variables influence the performance of the cameras in a real setup. Specific light-source power distributions might favor a particular camera hindering the comparability. A mathematical description of noise might not be an adequate descriptions of practical noise behavior of a physical camera. A chosen noise model could also favor one camera for the comparison.
This model can be inverted for spectral reconstruction, by estimating R j ( λ ) . Several different techniques have been proposed including the pseudo-inverse method [44] (linear least-square fitting) or linear least-square fitting in lower-dimensional space (Imai–Berns method) [45]. For this study, a commonly used spectral reconstruction technique known as Wiener estimation [45,46,47,48,49,50] is applied. Before inverting Equation (1) it is rewritten into discrete formulation:
P i = ∑ k = 0 N E ( λ k ) R j ( λ k ) Q i ( λ k )
N is the number of spectral bands depending on the wavelength range and spectral resolution, in this case, λ ∈ [ 400 , 700 ] with a sampling rate of 2 nm steps and N = 151 . For all j reflectances of the training set, the channels i of the sensor and k distinct spectral bands, we can write in matrix form:
p = R E Q
p is of J × I dimensionality with J spectral samples and I channels, R of J × N , E of N × N (diagonal matrix) and Q of N × I where N is 151 different wavelengths for this research. This is inverted according to the Wiener estimation method [45,46,47], in this study the implementation by Nishidate et al. [49] is followed and describes a reconstructed reflectance with:
r ˜ = Wp ,
where W describes the Wiener estimation matrix, r ˜ the resulting vector of reflectance estimation or reconstruction and p the vector of sensor responses for each channel. The Wiener matrix is calculated by minimizing the square error of reconstructed and given reflectance for a training set of reflectances.
This matrix needs to be calculated for each camera and training set combination. Sensor responses can be simulated by multiplying the sensor sensitivities and the reflectance spectrum of an object. Spectral reconstructions can then be performed given this sensor response and the pre-trained Wiener estimation matrix W .

3.2. Sensors

Most SFA sensors are based on micro interference filters (often Fabry–Pérot interference) that can be simulated with GSB as shown by Lapray et al. [51] with width and shape as main parameters [52,53].
The framework enables the comparison of any multi-band sensors with known spectral sensitivities or optimize the design of ’virtual’ SFA cameras for specific applications. SFA cameras have a limited number of wavelength bands divided over the sensor. The design of SFA sensors will be a trade-off between spectral resolution and spectral range covered. A narrower spectral band per filter will improve the spectral resolution, but would require more spectral bands to cover the whole sensitivity. Broader sensitivities on the other hand, reduce the spectral resolution, but require less filters and avoid (“holes”) in the covered spectrum. However, for specific applications only a few primary wavelengths are needed as in case of oxygenation estimation.
In this study, we included simulated GSB they were chosen with a full width half max that make them comparable with them real sensor sensitivities of the cameras tested.

3.3. Training and Test Set

The training set will contribute to the accuracy of spectral reconstruction using Wiener estimation which calculates a transformation matrix that translates SFA responses to a full spectrum. This transformation matrix should minimize the difference between the reference spectrum and a reconstructed spectrum. The reference spectrum used to determine this matrix is called the training set.
For training two sets were compared to see the impact on the reconstruction accuracy for the different cameras: The Munsell database is used as a standard for color testing and the second training set was a generated for skin color simulation using a wide array of skin optical properties. The skin simulation (training set) assumes an equi-energy illumination and therefore represents illumination corrected skin spectra. Both sets are normalized using a feature scaling so that all values cover a range from 0 to 1. A more detailed description of this skin database follows in the experimental setup. For the validation if the spectral reconstruction another set based on skin reflectances was used. These skin reflectances (test set) are measured using a spectrophotometer and illumination corrected as described in [42].
The three sets are illustrated in Figure 2. This Figure allows comparison of the area covered by all sets and highlights three reflectances for each dataset. It includes the database of 100 measured skin reflectances [42], 10000 Monte Carlo simulated reflectances and the Munsell reflectances color patches [40,41].

3.4. Evaluation Metrics

The validation of the proposed framework can be tested by applying it to a specific clinical application, oxygen level estimation. This should show which spectral filter array camera is most suitable for this specific application. Three different evaluation metrics are considered. Two of the metrics focus on spectral reconstruction quality regarding shape and color. The third metric is application-specific and in this case quantifies the ability of each camera to estimate oxygen levels, it will be discussed in detail in the next section.
The first metric calculates the color difference Δ E 00 [43] of two spectra which is the distance between two colors in the human perceptual colorspace. Each spectrum is converted into color coordinates using the, D65 illumination for the calculations, and CIE 1931 2 Degree Standard Observer color-matching functions. A Δ E 00 of around 2 is a just noticeable color difference for a human observer.
The second spectral reconstruction metric is the root mean square error (RMSE) between the reference spectrum and a reconstructed spectrum. There is no need to include the goodness of fit coefficient (GFC) or the angular error, since previous studies [54] have shown that these correlate strongly with the RMSE.

3.5. Application-Specific Metric and Oxygenation Level Estimation

The third metric is a validation of the oxygenation level estimations. This parameter can be approximated through calculations using the reflectance spectrum of skin. The reflectance spectrum of skin is the result of concentrations of particular chromophores present in the skin. The ratio between oxygenated and deoxygenated hemoglobin reflects the relative oxygenation level in the skin and is an important parameter for diagnostics. Hemoglobin occurs in different forms but only these two are relevant for oxygenation. Different methods have been proposed to estimate oxygenation levels from particular wavelengths [27,29,49,55].
For this study, the estimation uses a multiple regression method described by Nishidate et al. [49]. A fast way of estimating absorbance A ( λ ) from reflectance assumes the Lambert-Beer law:
A ( λ ) = − l o g 10 R ( λ )
According to the simplified Lambert-Beer law the total absorbance of skin tissue can be described with:
A ( λ ) = C m l e ( λ , C m ) ε m ( λ ) + C b i l d ( λ , C b i ) ε b i ( λ ) + C o b l d ( λ , C o b , C d b ) ε o b ( λ ) + C d b l d ( λ , C o b , C d b ) ε d b ( λ ) ,
where ε m , ε b , ε o b , ε d b describe the molar extinction coefficients of melanin, bilirubin, oxygenated and deoxygenated hemoglobin and C m , C b , C o b , C d b describe the concentration of each specific chromophore. l e describes the mean optical path length for epidermis, l d for dermis and D ( λ ) describes the attenuation due to scattering these values are taken from literature. This equation can be solved by multiple regression analysis and is therefore reformulated to:
A ( λ 1 ) = c m ε m ( λ 1 ) + c b i ε b i ( λ 1 ) + c o b ε o b ( λ 1 ) + c d b ε d b ( λ 1 ) A ( λ 2 ) = c m ε m ( λ 2 ) + c b i ε b i ( λ 2 ) + c o b ε o b ( λ 2 ) + c d b ε d b ( λ 2 ) A ( λ 3 ) = c m ε m ( λ 3 ) + c b i ε b i ( λ 3 ) + c o b ε o b ( λ 3 ) + c d b ε d b ( λ 3 ) . . A ( λ n ) = c m ε m ( λ n ) + c b i ε b i ( λ n ) + c o b ε o b ( λ n ) + c d b ε d b ( λ n ) ,
where c m , c b i , c o b , c d b are closely related to the concentrations of melanin, bilirubin, oxygenated and deoxygenated blood and represent the unit-less contribution of each extinction coefficient to the total absorbance A. Any number of wavelengths can be used to calculate the absorbances. Reflectance spectra can be converted to absorbance spectra according to Equation (5) and then used with the following equation. The calculation of the concentration of any chromophore can then be formulated in matrix notation as:
a = ε c c = ε − 1 a
Finally, oxygen saturation can be calculated with:
S o x y = C o b C o b + C d b
Even though a simplification of the physical light skin interactions, methods based on these principles have been used for oxygenation level estimation [49,56,57,58]. This approach allows rapid calculation of tissue parameters with low computational complexity. It is assumed that most other chromophores are constant over time. The oxygenation of blood is not constant, due to oxygen consumption by tissue. According to Equation (9) oxygenation level estimation is calculated using both the reflectance spectra and reconstructed spectra. The Euclidean distance between the two resulting oxygenation level estimation values is then calculated and used as a quality metric to judge the reconstruction accuracy with:
Δ O x y = ∣ S o x y 1 − S o x y 2 ∣

4. Experimental Setup

This section will be discussing the concrete choices of sensors, training and test data, and finally, summarize the approach. A new database of simulated skin spectra is also created and described in detail in this section.

4.1. Sensors

Five cameras are investigated the Sinarback 54 RGB camera (RGB) as representative for common three-channel imaging, three spectral filter array cameras are considered, XIMEA xiSpec MQ022HG-IM-SM4X4-VIS [4,5], Silios technologies CMC-C [6] (Silios) and a prototypical device by Thomas et al. [3] (France1). Table 1 provides an overview of their key features and is sorted by the number of bands. The CorXim ’virtual cameras is added, which is the corrected version of the Ximea xispec [4] camera by applying a linear transformation matrix provided by the manufacturer which reduces the effect of secondary transmission peaks in some filter bands [59]. It is considered to be an independent camera to test the impact of such a correction.
Figure 3 and Figure 4 show the spectral sensitivities of all cameras in the spectral range of (400–700 nm) with a measurement interval of 2 nm steps. All sensitivities are measured and provided by the camera manufacturers and interpolated to this range and measurement interval. Additionally, for each camera a virtual GSB sensor is generated and included in the study. The GSB are generated according to Thomas [52] at each of the sensitivity peaks of each camera(GRGB, GFrance1, GSilios, GCorXim, GXimea). All GSB have a σ = 15 nm and provide a virtual version of each camera with perfectly shaped narrow band sensitivities.

4.2. Generating a Training Set

The skin simulations are generated using a modification of the multi-layered Monte Carlo tissue model (MCML) published by Atencio et al. [39]. This code was modified to vary and simulate combinations chromophore concentrations and blood volume fractions [61]. Changing Bilirubin concentration C b i , oxygen saturation S o x y , blood and melanin volume fractions f b l and f m e l were changed and 10,000 skin reflectances were calculated.
This simulation environment is based on a three-layer skin model and initially proposed to simulate bilirubin concentration in the skin of the forehead of newborns. The three layers are epidermis, dermis and a bone layer. This model assumes each of the layers as infinite homogenous media with a defined absorption per layer. Scattering is assigned uniformly to both layers. Each layer has different chromophores contributing to its absorption based on the volume fractions of melanin ( f m e l ) blood, ( f b l ) and bilirubin ( f b i ). Epidermis contains melanin, dermis contains bilirubin and oxy- deoxygenated hemoglobin. The total absorbance of each of the layers is the sum of the absorbance fractions of chromophores present in that particular layer and defined as μ a . The chromophore parameters for the Monte Carlo simulation, were chosen to cover the entire range defined by Atencio et al. [39] (see Table 2). For melanin volume fractions of approximately 1% to 6.3% equivalent to fair skin according to Jacques [62].
Each of the chromophore absorption coefficients is modelled from the data provided by Jacques et al. [63]. This μ a can be seen as analogous to the Absorption A in previous equations, but in the context of defining optical properties of skin it is referred to as μ a . For the epidermis, the absorption μ a e only depends on melanin the only chromophore present in this layer with:
μ a e p i = f m e l μ a m e l ( λ )
The Monte Carlo simulation framework by Atencio et al. mentions specifically that the model needs further testing and verification to simulate darker skin types, therefore even higher melanin volume fractions were not included as parameters for the simulation. To calculate the final absorption of the dermis layer both bilirubin and blood are the main contributors:
μ a d e r m = f b l μ a b l ( λ ) + f b i μ a b i ( λ )
f b i is considered to be constant and the parameter is the concentration of bilirubin as:
μ a b i l ( λ ) = l n ( 10 ) C b i P M b i ϵ b i ( λ ) ,
where P M b i is a constant and ϵ b i ( λ ) are the literature values for the extinction coefficients for bilirubin [63]. f b l describes the volume fraction of total blood in the dermis layer. The volume fraction parameters for this simulation cover typical values homogeneously distributed blood in the dermis layer [63]. Due to differences in absorbance for oxygenated hemoglobin and deoxygenated hemoglobin μ a b l o ( λ ) is calculated as:
μ a b l o = S μ a o b ( λ ) + ( 1 − S ) μ a d b ( λ )
S describes the oxygen saturation in the blood and is to be estimated. The dataset will be verified in the Results Section 5 using a principle component analysis.

5. Results and Discussion

5.1. Training Set Validation

The first results presented in this study address the skin simulation database and can be seen as an additional verification for using this simulated training set. It is based on principle component analysis (PCA) of the sets included in this research.
The principle components allow representation of the multidimensional set in a lower-dimensional space. If the principle components are calculated for a combined set they represent the orthogonal axes of a space describing the sets. The area covered by the sets plot into this orthogonal space describes the diversity of the particular set. If multiple sets are plot into the same principle component space the difference in diversity and area covered within that PCA space can be analyzed.
The sets are shown along the first two principal components of the combined set in Figure 5. Table 3 shows the resulting principle components of each of the sets and the combined set. The Munsell set is the most diverse considering its low first principle component. The skin simulation set covers a wider range of reflectances compared to measured skin reflectances. This is represented in a lower first principle component. Physiological parameters cover a wider range than living tissue see Table 2.
In Figure 5 it can be observed that the skin simulation covers all the measured skin reflectances except for a few measurements. This can be ascribed to the limited number of parameters for the simulation, resulting in some measured skin reflectances not being represented within the skin simulation. The skin model is limited to Caucasian fair skin and initially designed for neonatal babies. To further analyze the parameter of the skin simulation, which falls far out of the measured skin reflectance, the extreme curves where plotted.
Figure 5D,E shows these extreme curves of both the skin reflectance and the skin simulation set as marked in Figure 5A. In Table 4 it becomes apparent that the main factor for the simulations is the blood volume parameter. All extreme results according to the PCA analysis have an extreme value for the blood volume. The melanin parameter also contributes to extreme values within the principle component space indicating the strong influence of melanin on the resulting skin spectra. In this principle component space, the bilirubin concentration parameter spreads the distribution of points.
Figure 5 also contains sRGB [64] color swatches reproduced under a virtual D65 illumination. These provide a visual impression of the color of the extreme points in the principle component space. They show that the extreme value curves, not included within the skin simulation represent darker skin types and that extreme values of the skin simulation can include physiologically unlikely scenarios of grey skin.

5.2. Spectral Reconstruction

Results for the two spectral reconstruction metrics calculated for each of the four sensors and their simulated GSB versions are shown in Figure 6. Each of the graphs shows mean results and standard deviation of the actual sensor as a circle and the GSB sensor results as a cross. All metrics are calculated with the different training sets (Munsell and skin simulation) for the spectral reconstruction and plotted. The cartesian coordinate system consists of the number of channels on the x-axes and the value for each of the metrics on the y-axes.
These plots allow the comparison between the sensors according to the different metrics in two scenarios. It can be observed that the performance in RMSE and Δ E 00 correspond to each other.
Figure 6 provides a plot of the Δ E 00 difference between the test reflectances and their reconstruction. Surprisingly, the plots show that the corrected Ximea performs the worst in the case of Munsell patches for training and according to Δ E 00 . This can be ascribed to the cut of spectral sensitivity imposed by the linear correction transformation. Figure 4 shows the low sensitivity of this sensor at the edges of the chosen spectral range (400 nm to 700 nm).
Figure 7 contains plots of the spectral reflectances ground-truth and reconstructed that are responsible for the highest Δ E 00 results for the corrected and uncorrected Ximea camera. The plot allows appreciation of the areas of the spectra that cause high Δ E 00 results. In the case of the corrected Ximea camera spectral regions that have low or zero sensitivity are wrongly reconstructed. This is not surprising but confirms the poorer performance of the corrected Ximea camera in comparison with the uncorrected Ximea camera in the Δ E 00 and RMSE metric. The more limited spectral coverage of the corrected spectral imager negatively influences the spectral reconstruction ability of this camera.
The second worst performer regarding color differences (Mean Δ E 00 = ~14 and Mean Δ E 00 = ~12) is the RGB camera. Both the low number of channels and their specific overlap in the spectral region seems to influence the estimation accuracy negatively. The lower performance of the GSB version can be ascribed to the low sigma ( σ = 15 ) of the gaussian filters. In the case of the RGB sensor, the coverage of the spectral range of interested is as seen in Figure 3 not optimal. The spectral distribution shows significant areas of very low spectral sensitivity and negatively influences the spectral reconstructions.
Both corrected (CorXim) and uncorrected Ximea benefit greatly from GSB improving the performance according to the Δ E 00 metric. For the Silios camera, the GSB only improve the Δ E 00 performance when using the expert training set as the skin simulation set. One explanation could be the sharp cut off for the GSB resulting from the bands that exceed the spectral range of this analysis. The prototypical sensor France1 has initially already close to gaussian sensitivities and does not benefit from the GSB.
The RMSE metric shows a similar trend compared to Δ E 00 . The Ximea camera scores better results regarding the RMSE in comparison with Δ E 00 . Differences between original sensors and GSB sensors are smaller considering this metric.
Training theWiener estimation matrix with the proposed specialized skin simulation set results in a more robust reconstruction according to Δ E 00 and RMSE for all tested cameras. The more general Munsell set lacks skin spectral shapes and is contains two dissimilar spectra in comparison with the skin test set. The similar shapes and increased number of spectra in the generated specialized database improve the spectral reconstructions.

5.3. Oxygenation Level Estimation

The oxygenation estimations were performed using six wavelengths as proposed by Nishidate et al. (500 nm, 520 nm, 540 nm, 560 nm, 580 nm, 600 nm) and three wavelengths (480 nm, 560 nm, 600 nm) the results are shown in Figure 8.
These two oxygenation metrics show different behavior for all cameras compared to the spectral accuracy metrics. The eight and nine channel cameras (France1 and Silios) perform the worst for the Munsell training case and six wavelengths. This is surprising since these two cameras perform the best according to the spectral reconstruction metrics Δ E 00 and RMSE. For this case, the performance differences between the GSB sensor and the original sensor are very small. One explanation can be that these key wavelengths all fall into valleys between the sensitivity peaks for the Silios and France1 sensor. The GSB sensors could be affected equally or stronger, due to the relatively small sigma ( σ = 15 ).
The wavelengths proposed by Nishidate et al. are optimized for an RGB sensor. For the specialized training set, the RGB camera performs the worst. Illustrating that the spectral reconstruction using a specialized training set benefits from narrow spectral channels.
Figure 8 also contains results for the oxygenation metric using three wavelengths (480 nm, 560 nm, 600 nm). It can be observed that the choice of the training set for this configuration influences the different cameras independently. For Munsell patch training, the RGB camera performs the worst and both versions of the Ximea camera the best. Using the specialized training set the differences between all cameras are smaller and the RGB camera still performs worst. The other sensors are less affected by the change of training sets only slightly lowering their oxygenation metric differences when using the specialized training set. For the idealized GSB RGB sensor lower oxygenation metric differences can be observed compared to some of the SFA sensors. This could be ascribed to the wavelength chosen for oxygenation level estimation which all fall well within high sensitivity of the gaussian RGB (GRGB) sensor.
A camera with sensitivity peaks at the wavelength of interest should perform optimally. This can be used if the wavelength of interest are known. None of the investigated cameras has optimal filter sensitivity peaks for oxygenation estimation. Table 5 provides an overview of the statistical results for all sensors, considering the better performing skin simulation training data set.
The proposed specialized training set improved the final oxygenation parameters (estimated with three wavelengths). In the case of six wavelengths the skin training set performs worse than the Munsell set. One explanation is that using six wavelengths includes wavelengths at the outer edges of the considered spectral range. The specialized set provides too little variety for these areas and the diverse Munsell set trains these regions better.
For future work noise should be incorporated into the framework. The chosen wiener estimation method has room to incorporate a noise term into the spectral estimation and the impact of different kind of noise should be studied. The framework also allows simulation and comparison of spectral filter array cameras in different spectral ranges. Near infrared should be considered for future work as it is used in traditional oximetry systems. Furthermore, oxygenation estimation methods that use the full spectra based on inverse Monte Carlo methods should be tested in conjunction with spectral reflectance reconstruction.

5.4. Summary and Conclusions

A straightforward framework to evaluate spectral filter array cameras based on spectral sensitivities and publicly available skin and reflectance databases was proposed. It allows to compare and quantify the performance of SFA cameras for medical applications and skin imaging in particular. The framework does not require prior measurements and is based on a readily available skin databases for testing, a proposed generated skin simulation database and sensor sensitivities of the cameras included.
Reconstructing full reflectances from sensor responses allows to comparison and is useful when the application-specific bands of interest are unknown. It can be useful to recreate color images and benefits from a specialized training set. If the bands of interest are known a camera with high sensitivity for those exact bands is advisable. Several observations particular to spectral filter array cameras were made:
  • Spectral shapes of the filters should be adapted application-specific
  • Careful choice of the spectral bands should be adapted application-specific
  • Selecting an optimal training set for spectral reflectances reconstruction improves the results for SFAs with narrow spectral sensitivities
  • GSB improve spectral reconstruction considering Δ E 00 color differences and RMSE
  • GSB have a small impact on oxygenation level estimation if the bands are not close to the ideal wavelength for oxygen estimation
The framework has been applied to compare commercially available SFA cameras for skin diagnosis and skin oxygenation level estimation.
The corrected Ximea camera performed the best in terms of oxygenation level estimations. Regarding the spectral reconstruction and Δ E 00 color difference metrics the Silios camera shows the best results. Recommending it for applications where the specific bands of interest are not known.
SFA cameras hold great potential for monitoring vital functions and medical diagnosis as a non-contact, real-time spectral imaging modality. This framework provides a basis for using spectral filter array cameras effectively for medical applications. It can be used to design spectral filter sensitivities for specific applications by optimizing the wavelength bands and transmission shapes of the filters. It is, however, necessary to verify the findings with experimental data and extend the framework to include spatial aspects.

Author Contributions

Conceptualization, J.R.B., J.-B.T., J.Y.H. and R.M.V.; Data curation, J.R.B.; Formal analysis, J.R.B.; Investigation, J.R.B.; Methodology, J.R.B. and J.-B.T.; Project administration, J.R.B., J.Y.H. and R.M.V.; Resources, J.Y.H. and R.M.V.; Software, J.R.B.; Supervision, J.-B.T., J.Y.H. and R.M.V.; Validation, J.R.B., J.-B.T., J.Y.H. and R.M.V.; Visualization, J.R.B. and J.-B.T.; Writing—original draft, J.R.B; Writing—review & editing, J.R.B., J.-B.T., J.Y.H. and R.M.V.

Funding

This research has been supported by the Research Council of Norway through project no. 247689 “IQ-MED: Image Quality enhancement in MEDical diagnosis, monitoring and treatment”.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
SFAspectral filter array
GFCgoodness of fit coefficient
RMSEroot mean square error
sRGBstandard RGB
MCMLMonte Carlo modelling of light transport in multi-layered tissues
GSBgaussian spectral bands

References

  1. Lapray, P.J.; Wang, X.; Thomas, J.B.; Gouton, P. Multispectral filter arrays: Recent advances and practical implementation. Sensors 2014, 14, 21626–21659. [Google Scholar] [CrossRef] [PubMed]
  2. Ewerlöf, M.; Larsson, M.; Salerud, E.G. Spatial and temporal skin blood volume and saturation estimation using a multispectral snapshot imaging camera. Proc. SPIE 2017. [Google Scholar] [CrossRef]
  3. Thomas, J.B.; Lapray, P.J.; Gouton, P.; Clerc, C. Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition. Sensors 2016, 16, 993. [Google Scholar] [CrossRef] [PubMed]
  4. Ximea. Hyperspectral Cameras. 2018. Available online: https://www.ximea.com (accessed on 2 December 2018).
  5. IMEC. Hyperspectral-Imaging. 2018. Available online: https://www.imec-int.com (accessed on 2 December 2018).
  6. SILIOS. Multispectral-Imaging. 2018. Available online: https://www.silios.com (accessed on 2 December 2018).
  7. Pedersen, M.; Hardeberg, J.Y. Full-reference image quality metrics: Classification and evaluation. Found. Trends® Comput. Graph. Vis. 2012, 7, 1–80. [Google Scholar] [CrossRef]
  8. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef] [Green Version]
  9. Chandler, D.M. Seven Challenges in Image Quality Assessment: Past, Present, and Future Research. ISRN Signal Process. 2013, 2013, 1–53. [Google Scholar] [CrossRef]
  10. Miao, L.; Qi, H.; Ramanath, R.; Snyder, W.E. Binary tree-based generic demosaicking algorithm for multispectral filter arrays. IEEE Trans. Image Process. 2006, 15. [Google Scholar] [CrossRef]
  11. Monno, Y.; Tanaka, M.; Okutomi, M. Multispectral demosaicking using adaptive kernel upsampling. In Proceedings of the 2011 18th IEEE International Conference on Image Processing, Brussels, Belgium, 11–14 September 2011; pp. 3157–3160. [Google Scholar]
  12. Wang, C.; Wang, X.; Hardeberg, J.Y. A Linear Interpolation Algorithm for Spectral Filter Array Demosaicking. In Image and Signal Processing; Springer International Publishing: Cham, Switzerland, 2014; pp. 151–160. [Google Scholar] [Green Version]
  13. Wang, X.; Thomas, J.B.; Hardeberg, J.Y.; Gouton, P. A Study on the Impact of Spectral Characteristics of Filters on Multispectral Image Acquisition. In Proceedings of the 12th Congress of the International Colour Association, Newcastle Gateshead, UK, 8–12 July 2013; Volume 4, pp. 1765–1768. [Google Scholar]
  14. Park, C.; Kang, M. Color Restoration of RGBN Multispectral Filter Array Sensor Images Based on Spectral Decomposition. Sensors 2016, 16, 719. [Google Scholar] [CrossRef]
  15. Nazari, R.M. Denoising and Demosaicking of Color Images. Ph.D. Thesis, Université d’Ottawa/University of Ottawa, Ottawa, ON, Canada, 2017. [Google Scholar]
  16. Bersha, K.S. Spectral Imaging and Analysis of Human Skin. Master’s Thesis, University of Estern Finland, Joensuu, Finnland, 2010. [Google Scholar]
  17. Kuzmina, I.; Diebele, I.; Jakovels, D.; Spigulis, J.; Valeine, L.; Kapostinsh, J.; Berzina, A. Towards noncontact skin melanoma selection by multispectral imaging analysis. J. Biomed. Opt. 2011, 16, 060502. [Google Scholar] [CrossRef]
  18. Nishidate, I.; Tanaka, N.; Kawase, T.; Maeda, T.; Yuasa, T.; Aizu, Y.; Yuasa, T.; Niizeki, K. Noninvasive imaging of human skin hemodynamics using a digital red-green-blue camera. J. Biomed. Opt. 2011, 16, 086012. [Google Scholar] [CrossRef]
  19. Jakovels, D.; Spigulis, J. RGB imaging device for mapping and monitoring of hemoglobin distribution in skin. Lith. J. Phys. 2012, 52, 50–54. [Google Scholar] [CrossRef]
  20. Jakovels, D.; Kuzmina, I.; Berzina, A.; Spigulis, J. RGB imaging system for monitoring of skin vascular malformation’s laser therapy. Proc. SPIE 2012, 8427, 842737. [Google Scholar]
  21. Kumar, A.; Dhawan, A.P.; Relue, P.; Chaudhuri, P.K. Multi-spectral optical imaging of skin to diagnose malignant melanoma. In Proceedings of the Engineering in Medicine and Biology, Atlanta, GA, USA, 13–16 October 1999; Volume 2, p. 1098. [Google Scholar]
  22. Cotton, S.; Claridge, E.; Hall, P. A skin imaging method based on a colour formation model and its application to the diagnosis of pigmented skin lesions. In Proceedings of the Medical Image Understanding and Analysis, BMVA, Oxford, UK, 19–20th July 1999; pp. 49–52. [Google Scholar]
  23. Tsumura, N.; Kawabuchi, M.; Haneishi, H.; Miyake, Y. Mapping Pigmentation in Human Skin by Multi-Visible-Spectral Imaging by Inverse Optical Scattering Technique. Color Imaging Conf. 2000, 2000, 81–84. [Google Scholar]
  24. Balas, C.; Themelis, G.; Papadakis, A.; Vasgiouraki, E. A novel hyper-spectral imaging system: Application on in-vivo detection and grading of cervical precancers and of pigmented skin lesions. In Proceedings of the IEEE Computer Society Workshop on Computer Vision Beyond the Visible Spectrum, Kauai, HI, USA, 14 December 2001. [Google Scholar]
  25. Kerekes, J.; Subramanian, N.; Kearney, K.; Schad, N. Spectral imaging of skin: Experimental observations and analyses. Proc. SPIE 2006, 6142, 61423V. [Google Scholar]
  26. Randeberg, L.L.; Baarstad, I.; Løke, T.; Kaspersen, P.; Svaasand, L.O. Hyperspectral imaging of bruised skin. Proc. SPIE 2006, 6078, 6078. [Google Scholar] [CrossRef]
  27. Klaessens, J.H.G.M.; Noordmans, H.J.; de Roode, R.; Verdaasdonk, R.M. Non-invasive skin oxygenation imaging using a multi-spectral camera system: Effectiveness of various concentration algorithms applied on human skin. Proc. SPIE 2009, 7174. [Google Scholar] [CrossRef]
  28. Spigulis, J.; Jakovels, D.; Rubins, U. Multi-spectral skin imaging by a consumer photo-camera. Proc. SPIE 2010, 7557. [Google Scholar] [CrossRef]
  29. Huang, J. Multispectral Imaging of Skin Oxygenation. Ph.D. Thesis, The Ohio State University, Columbus, OH, USA, 2013. [Google Scholar]
  30. Poxon, I.; Wilkinson, J.; Herrick, A.; Dickinson, M.; Murray, A. Pilot study to visualise and measure skin tissue oxygenation, erythema, total haemoglobin and melanin content using index maps in healthy controls. Proc. SPIE 2014, 8951, 89510X. [Google Scholar]
  31. Van Gastel, M.; Stuijk, S.; De Haan, G. New principle for measuring arterial blood oxygenation, enabling motion-robust remote monitoring. Sci. Rep. 2016, 6, 38609. [Google Scholar] [CrossRef] [Green Version]
  32. Bauer, J.R.; van Beekum, K.; Klaessens, J.H.G.M.; Noordmans, H.J.; Boer, C.; Hardeberg, J.Y.; Verdaasdonk, R.M. Towards real-time non contact spatial resolved oxygenation monitoring using a multi spectral filter array camera in various light conditions. Proc. SPIE 2018, 10489. [Google Scholar] [CrossRef]
  33. Preece, S.J.; Claridge, E. Spectral filter optimization for the recovery of parameters which describe human skin. IEEE Trans. Pattern Anal. Mach. Intell. 2004, 26, 913–922. [Google Scholar] [CrossRef] [PubMed]
  34. Gutiérrez-Gutiérrez, J.; Pardo, A.; Real, E.; López-Higuera, J.; Conde, O.M. Custom Scanning Hyperspectral Imaging System for Biomedical Applications: Modeling, Benchmarking, and Specifications. Sensors 2019, 19, 1692. [Google Scholar] [CrossRef] [PubMed]
  35. Saager, R.B.; Baldado, M.L.; Rowland, R.A.; Kelly, K.M.; Durkin, A.J. Method using in vivo quantitative spectroscopy to guide design and optimization of low-cost, compact clinical imaging devices: Emulation and evaluation of multispectral imaging systems. J. Biomed. Opt. 2018, 23, 1–12. [Google Scholar] [CrossRef] [PubMed]
  36. Jimenez, J.; Scully, T.; Barbosa, N.; Donner, C.; Alvarez, X.; Vieira, T.; Matts, P.; Orvalho, V.; Gutierrez, D.; Weyrich, T. A practical appearance model for dynamic facial color. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 2010, 29, 141:1–141:10. [Google Scholar]
  37. Iglesias-Guitian, J.A.; Aliaga, C.; Jarabo, A.; Gutierrez, D. A Biophysically-Based Model of the Optical Properties of Skin Aging. Comput. Graph. Forum 2015, 34, 45–55. [Google Scholar] [CrossRef]
  38. Lapray, P.J.; Thomas, J.B.; Gouton, P. High Dynamic Range Spectral Imaging Pipeline For Multispectral Filter Array Cameras. Sensors 2017, 17, 1281. [Google Scholar] [CrossRef] [PubMed]
  39. Delgado Atencio, J.A.; Jacques, S.L.; Montiel, S.V. Monte Carlo Modeling of Light Propagation in Neonatal Skin; InTech: London, UK, 2011. [Google Scholar]
  40. Hiltunen, J. Munsell Book of Color: Matte Finish Collection Measured by J. Hiltunen. 2019. Available online: https://www.uef.fi/web/spectral/munsell-colors-matt-spectrofotometer-measured (accessed on 2 December 2018).
  41. Munsell Color. Munsell Book of Color: Matte Finish Collection; Munsell Color: Baltimore, MD, USA, 1976. [Google Scholar]
  42. Cooksey, C.C.; Allen, D.W.; Tsai, B.K. Reference Data Set of Human Skin Reflectance. J. Res. Natl. Inst. Stand. Technol. 2017, 122, 1–5. [Google Scholar] [CrossRef]
  43. Publication, CIE. CIE 15: Technical Report: Colorimetry, 3rd ed.; CIE Cent. Bur.: Vienna, Austria, 2004; Volume 3. [Google Scholar]
  44. Hardeberg, J.Y. Acquisition and Reproduction of Color Images: Colorimetric and Multispectral Approaches. Ph.D. Thesis, Ecole Nationale Supérieure des Télécommunications, Paris, France, 1999. [Google Scholar]
  45. Imai, F.H.; Berns, R.S. Spectral estimation using trichromatic digital cameras. In Proceedings of the International Symposium on Multispectral Imaging and Color Reproduction for Digital Archives, Chiba, Japan, 21–22 October 1999; Volume 42, pp. 1–8. [Google Scholar]
  46. Shimano, N.; Terai, K.; Hironaga, M. Recovery of spectral reflectances of objects being imaged by multispectral cameras. J. Opt. Soc. Am. A 2007, 24, 3211. [Google Scholar] [CrossRef]
  47. Shimano, N.; Hironaga, M. Recovery of spectral reflectances of imaged objects by the use of features of spectral reflectances. J. Opt. Soc. Am. A 2010, 27, 251–258. [Google Scholar] [CrossRef]
  48. Stigell, P.; Miyata, K.; Hauta-Kasari, M. Wiener estimation method in estimating of spectral reflectance from RGB images. Pattern Recognit. Image Anal. 2007, 17, 233–242. [Google Scholar] [CrossRef]
  49. Nishidate, I.; Maeda, T.; Niizeki, K.; Aizu, Y. Estimation of Melanin and Hemoglobin Using Spectral Reflectance Images Reconstructed from a Digital RGB Image by the Wiener Estimation Method. Sensors 2013, 13, 7902–7915. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  50. Heikkinen, V.; Lenz, R.; Jetsu, T.; Parkkinen, J.; Hauta-Kasari, M.; Jääskeläinen, T. Evaluation and unification of some methods for estimating reflectance spectra from RGB images. J. Opt. Soc. Am. A 2008, 25, 2444–2458. [Google Scholar] [CrossRef] [PubMed]
  51. Lapray, P.J.; Thomas, J.B.; Gouton, P.; Ruichek, Y. Energy balance in Spectral Filter Array camera design. J. Eur. Opt. Soc.-Rapid Publ. 2017, 13, 1. [Google Scholar] [CrossRef]
  52. Thomas, J.B. Illuminant estimation from uncalibrated multispectral images. In Proceedings of the 2015 Colour and Visual Computing Symposium (CVCS), Gjøvik, Norway, 25–26 August 2015; pp. 1–6. [Google Scholar] [CrossRef]
  53. Wang, X.; Thomas, J.B.; Hardeberg, J.Y.; Gouton, P. Multispectral imaging: Narrow or wide band filters? JAIC J. Int. Colour Assoc. 2014, 12, 44–51. [Google Scholar]
  54. Khan, H.A.; Thomas, J.B.; Hardeberg, J.Y.; Laligant, O. Illuminant estimation in multispectral imaging. J. Opt. Soc. Am. A 2017, 34, 1085–1098. [Google Scholar] [CrossRef]
  55. Randeberg, L.L.; Winnem, A.; Blindheim, S.; Haugen, O.; Svaasand, L. Optical classification of bruises. Proc. SPIE 2004, 5312, 54–64. [Google Scholar] [CrossRef]
  56. Humphreys, K.; Ward, T.; Markham, C. A CMOS camera-based pulse oximetry imaging system. In Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China, 17–18 January 2006; pp. 3494–3497. [Google Scholar]
  57. Kong, L.; Yi, D.; Sprigle, S.; Wang, F.; Wang, C.; Liu, F.; Adibi, A.; Tummala, R. Single sensor that outputs narrowband multispectral images. J. Biomed. Opt. 2010, 15. [Google Scholar] [CrossRef]
  58. Spigulis, J.; Oshina, I.; Berzina, A.; Bykov, A. Smartphone snapshot mapping of skin chromophores under triple-wavelength laser illumination. J. Biomed. Opt. 2017, 22, 091508. [Google Scholar] [CrossRef]
  59. Bauer, J.R.; Bruins, A.A.; Hardeberg, J.Y.; Verdaasdonk, R.M. A Spectral Filter Array Camera for Clinical Monitoring and Diagnosis: Proof of Concept for Skin Oxygenation Imaging. J. Imaging 2019, 5. [Google Scholar] [CrossRef]
  60. Day, D. Spectral Sensitivities of the Sinarback 54 Camera; Technical Report; Munsell Color Science Laboratory, Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology: Rochester, NY, USA, 2003. [Google Scholar]
  61. Bauer, J.R.; Pedersen, M.; Hardeberg, J.Y.; Verdaasdonk, R. Skin color simulation - review and analysis of available Monte Carlo-based photon transport simulation models. Color Imaging Conf. 2017, 2017, 165–170. [Google Scholar]
  62. Jacques, S. Origins of tissue optical properties in the UVA, visible, and NIR regions. Adv Opt Imaging Photon Migr. 1996, 2, 364–369. [Google Scholar]
  63. Jacques, S.L.; Prahl, S.A. A Collaboration of Oregon Health & Science University, Portland State University, and the Oregon Institute of Technology. Optical Spectra. 2015. Available online: https://www.omlc.org (accessed on 2 December 2018).
  64. IEC. International Standard: International Electrotechnical Commission; IEC 61966-2-1:1999; IEC: Geneva, Switzerland, 1999. [Google Scholar]
Sensors 19 04805 g001 550
Figure 1. Framework, including training set, testing set, sensor sensitivities, reconstructed spectra and the evaluation according to RMSE, Δ E 00 , Δ O x y .
Figure 1. Framework, including training set, testing set, sensor sensitivities, reconstructed spectra and the evaluation according to RMSE, Δ E 00 , Δ O x y .
Sensors 19 04805 g001
Sensors 19 04805 g002 550
Figure 2. Measured Munsell relfectances [40] (Munsell), measured skin reflectances [42] (SkinRef), simulated skin reflectances (SkinSim). Three reflectances highlighted for visibility in each set.
Figure 2. Measured Munsell relfectances [40] (Munsell), measured skin reflectances [42] (SkinRef), simulated skin reflectances (SkinSim). Three reflectances highlighted for visibility in each set.
Sensors 19 04805 g002
Sensors 19 04805 g003 550
Figure 3. Sensor sensitivities, one RGB camera [60], a prototypical implementation by Thomas et al. [3] (France1) and commercially available Silios [6] (Silios) (all left) and simulated GSB (GRGB, GFrance1 and GSilios) versions (all right).
Figure 3. Sensor sensitivities, one RGB camera [60], a prototypical implementation by Thomas et al. [3] (France1) and commercially available Silios [6] (Silios) (all left) and simulated GSB (GRGB, GFrance1 and GSilios) versions (all right).
Sensors 19 04805 g003
Sensors 19 04805 g004 550
Figure 4. Sensor sensitivities, Ximea xispec [4] (Ximea and CorXim) (all left) and simulated GSB (GCorXim and GXimea) versions (all right).
Figure 4. Sensor sensitivities, Ximea xispec [4] (Ximea and CorXim) (all left) and simulated GSB (GCorXim and GXimea) versions (all right).
Sensors 19 04805 g004
Sensors 19 04805 g005 550
Figure 5. Dimensionality analysis of all sets combined (B) skin simulation (blue), skin reflectance (green) and Munsell reflectances (red). Colored markings for maximum PCA1, minimum PCA1, maximum PCA2, minimum PCA2, for skin simulation and skin reflectance, respectively. Color patch recreation (under D65 light source) of the extreme spectra for the skin simulation (A) and skin reflectance database (C) with minimum PCA 1 and PCA 2 and maximum PCA 1 and PCA 2. Plot of the maximum and minimum spectra for the skin reflectance database (D) and skin simulation database (E) according to PCA analysis.
Figure 5. Dimensionality analysis of all sets combined (B) skin simulation (blue), skin reflectance (green) and Munsell reflectances (red). Colored markings for maximum PCA1, minimum PCA1, maximum PCA2, minimum PCA2, for skin simulation and skin reflectance, respectively. Color patch recreation (under D65 light source) of the extreme spectra for the skin simulation (A) and skin reflectance database (C) with minimum PCA 1 and PCA 2 and maximum PCA 1 and PCA 2. Plot of the maximum and minimum spectra for the skin reflectance database (D) and skin simulation database (E) according to PCA analysis.
Sensors 19 04805 g005
Sensors 19 04805 g006 550
Figure 6. Resulting metrics of Δ E 00 (D65, CIE 2 ∘ 1931) (top) and R M S E calculated between reconstruction and training (bottom). All Sensors, Munsell set (left) and Skin Simulation set (right) as training including standard deviation of the resulting data. For all graphs, the filled “o” represents the original sensor and the “x” represents the GSB.
Figure 6. Resulting metrics of Δ E 00 (D65, CIE 2 ∘ 1931) (top) and R M S E calculated between reconstruction and training (bottom). All Sensors, Munsell set (left) and Skin Simulation set (right) as training including standard deviation of the resulting data. For all graphs, the filled “o” represents the original sensor and the “x” represents the GSB.
Sensors 19 04805 g006
Sensors 19 04805 g007 550
Figure 7. Visualization of worst and best Δ E 00 results for the uncorrected Ximea (Ximea top) and corrected Ximea (CorXim bottom), Munsell set (left) and Skin simulation (SkinSim) set (right) as training. Each graph includes GSB sensor results, ground-truth in solid lines and estimation with dashed lines.
Figure 7. Visualization of worst and best Δ E 00 results for the uncorrected Ximea (Ximea top) and corrected Ximea (CorXim bottom), Munsell set (left) and Skin simulation (SkinSim) set (right) as training. Each graph includes GSB sensor results, ground-truth in solid lines and estimation with dashed lines.
Sensors 19 04805 g007
Sensors 19 04805 g008 550
Figure 8. Resulting values for Δ Oxy metric calculated using six wavelength (500 nm, 520 nm, 540 nm, 560 nm, 580 nm, 600 nm) (top) and three wavelengths (480 nm, 560 nm, 600 nm) (bottom) for all Sensors. Munsell set (left) as training and Skin Simulation set (right) including standard deviation of the data.
Figure 8. Resulting values for Δ Oxy metric calculated using six wavelength (500 nm, 520 nm, 540 nm, 560 nm, 580 nm, 600 nm) (top) and three wavelengths (480 nm, 560 nm, 600 nm) (bottom) for all Sensors. Munsell set (left) as training and Skin Simulation set (right) including standard deviation of the data.
Sensors 19 04805 g008
Table
Table 1. Features of the included cameras. RGB camera [60], commercially available XIMEA Xispec SFA camera [4,5] (Ximea and CorXim), Silios technologies SFA camera [6] (Silios), and a prototypical device from academia [3] (France1).
Table 1. Features of the included cameras. RGB camera [60], commercially available XIMEA Xispec SFA camera [4,5] (Ximea and CorXim), Silios technologies SFA camera [6] (Silios), and a prototypical device from academia [3] (France1).
PropertyRGBFrance1SiliosCorXimXimea
spectral bands3891016
spectral peak range [nm]480–610440–850445–710465–630465–630
frame rate [Hz]606060170170
resolution per band 4080 × 5440 160 × 128 426 × 339 512 × 272 512 × 272
size [mm] 38 . 8 × 50 . 0 NA 56 × 56 × 22 26 × 26 × 26 26 × 26 × 26
Table
Table 2. Parameter range for MCML (Monte Carlo modelling of light transport in multi-layered tissues) skin simulation [39,61] resulting in 10,000 different parameter combinations. S o x y is the saturation of oxygenation, f b l and f m e l the volume fraction of blood and bilirubin, and C b i describes the bilirubin concentration. Green and red Shadings indicate extreme values of simulation range.
Table 2. Parameter range for MCML (Monte Carlo modelling of light transport in multi-layered tissues) skin simulation [39,61] resulting in 10,000 different parameter combinations. S o x y is the saturation of oxygenation, f b l and f m e l the volume fraction of blood and bilirubin, and C b i describes the bilirubin concentration. Green and red Shadings indicate extreme values of simulation range.
ParameterLevel: 12345678910
S o x y 10%20%30%40%50%60%70%80%90%100%
f b l 0.1%0.2%0.3%0.4%0.5%0.6%0.7%0.8%0.9%1%
C b i 0.00.0250.050.0750.10.1250.150.1750.20.225
f m e l 02%3%4 %5%6%7%8%9%10%
Table
Table 3. Resulting PCAs for all sets. Variance of each of the sets along the first 4 principle components.
Table 3. Resulting PCAs for all sets. Variance of each of the sets along the first 4 principle components.
PCAMunsellSkinSimSkinReflCombined
176.887.196.074.7
215.87.72.117.0
36.04.31.54.8
40.80.50.22.1
Table
Table 4. Monte Carlo Simulation parameters for the extreme points according to the principle components. Red background items indicate the maximum of their particular parameter, while green background indicate minima for the range of input parameters.
Table 4. Monte Carlo Simulation parameters for the extreme points according to the principle components. Red background items indicate the maximum of their particular parameter, while green background indicate minima for the range of input parameters.
ParameterMax PCA1 and PCA2Min PCA1Min PCA2
S t O 2 S a t u r a t i o n 10%100 %10%
f B l o o d 0.1%1%1%
C B i l i r u b i n 0.2250.00.225
f M e l a n i n 0.0 %10%10%
Table
Table 5. Statistical results (minimum, maximum, mean, standard deviation, 98 % ) for all sensors for Δ E 00 (top), RMSE (2nd from top), Δ Oxy 6wvl (3rd from top) and Δ Oxy 3wvl (bottom). All values are based on skin simulation set as training and skin reflectance set as testing.
Table 5. Statistical results (minimum, maximum, mean, standard deviation, 98 % ) for all sensors for Δ E 00 (top), RMSE (2nd from top), Δ Oxy 6wvl (3rd from top) and Δ Oxy 3wvl (bottom). All values are based on skin simulation set as training and skin reflectance set as testing.
Δ E 00
SensorMinMaxMeanStd98% MinMaxMeanStd98%
RGB5.0411.037.271.089.20 GRGB8.8916.7612.011.5214.87
France10.020.930.220.150.68 GFrance10.401.500.860.231.35
Silios0.040.660.280.110.49 GSilios0.030.750.250.120.51
CorXim5.8211.998.741.2411.50 GCorXim0.022.270.510.442.18
Ximea0.896.814.401.166.38 GXimea0.000.300.090.070.25
RMSE
SensorMinMaxMeanStd98% MinMaxMeanStd98%
RGB0.0006470.0021120.0010990.0002630.001717 GRGB0.000670.001940.001080.000240.00180
France10.000010.000090.000040.000010.00007 GFrance10.000030.000370.000100.000050.00022
Silios0.0000030.000060.000030.000010.00005 GSilios0.0000030.000060.000030.000010.00006
CorrXim0.0001840.000990.000400.000140.00081 GCorXim0.0000040.000280.000040.000050.00026
Ximea0.0000070.000280.000100.000040.00020 GXimea0.0000020.000030.000010.000010.00003
Oxyg. Metric 6wvl
SensorMinMaxMeanStd98% MinMaxMeanStd98%
RGB0.0700.1690.1140.0200.155 GRGB0.0510.1250.0840.0180.119
France10.0010.1500.0400.0310.109 GFrance10.00010.1450.0330.0300.102
Silios0.0020.1400.0730.0310.131 GSilios0.0060.1510.0750.0320.134
CorXim0.0010.0280.0180.0050.027 GCorXim0.00020.0170.0070.0040.016
Ximea0.0000.0190.0090.0040.018 GXimea00.0020.0170.0090.0030.016
Oxyg. Metric 3wvl
SensorMinMaxMeanStd98% MinMaxMeanStd98%
RGB0.0100.1320.0510.0220.090 GRGB0.00010.0510.0140.0100.044
France10.0010.0410.0250.0080.038 GFrance10.0020.0480.0250.0100.043
Silios0.0010.0430.0190.0080.035 GSilios0.0010.0480.0190.0090.036
CorXim0.000040.0110.0060.0020.010 GCorXim0.00010.0080.0030.0020.007
Ximea0.0010.0410.0250.0080.038 GXimea0.000010.0070.0040.0010.006

Share and Cite

MDPI and ACS Style

Bauer, J.R.; Thomas, J.-B.; Hardeberg, J.Y.; Verdaasdonk, R.M. An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis. Sensors 2019, 19, 4805. https://doi.org/10.3390/s19214805

AMA Style

Bauer JR, Thomas J-B, Hardeberg JY, Verdaasdonk RM. An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis. Sensors. 2019; 19(21):4805. https://doi.org/10.3390/s19214805

Chicago/Turabian Style

Bauer, Jacob Renzo, Jean-Baptiste Thomas, Jon Yngve Hardeberg, and Rudolf M. Verdaasdonk. 2019. "An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis" Sensors 19, no. 21: 4805. https://doi.org/10.3390/s19214805

APA Style

Bauer, J. R., Thomas, J. -B., Hardeberg, J. Y., & Verdaasdonk, R. M. (2019). An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis. Sensors, 19(21), 4805. https://doi.org/10.3390/s19214805

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop