Diffuse reflectance spectroscopy

The mathematical treatments of absorption spectroscopy for scattering materials were originally largely borrowed from other fields. The most successful treatments use the concept of dividing a sample into layers, called plane parallel layers. The treatments are generally those consistent with a two-flux or two-stream approximation. Some of the treatments require all the scattered light, both remitted and transmitted light, to be measured. Others apply only to remitted light, with the assumption that the sample is "infinitely thick" and transmits no light. These are special cases of the more general treatments.

There are several general treatments, all of which are compatible with each other, related to the mathematics of plane parallel layers. They are the Stokes formulas,^[1] equations of Benford,^[2] Hecht finite difference formula,^[3] and the Dahm equation.^[4][5] For the special case of infinitesimal layers, the Kubelka–Munk^[6] and Schuster–Kortüm^[7][8] treatments also give compatible results. Treatments which involve different assumptions and which yield incompatible results are the Giovanelli^[9] exact solutions, and the particle theories of Melamed^[10] and Simmons.^[11]

George Gabriel Stokes (not to neglect the later work of Gustav Kirchhoff) is often given credit for having first enunciated the fundamental principles of spectroscopy. In 1862, Stokes published formulas for determining the quantities of light remitted and transmitted from "a pile of plates". He described his work as addressing a "mathematical problem of some interest". He solved the problem using summations of geometric series, but the results are expressed as continuous functions. This means that the results can be applied to fractional numbers of plates, though they have the intended meaning only for an integral number. The results below are presented in a form compatible with discontinuous functions.

Stokes used the term "reflexion", not "remission", specifically referring to what is often called regular or specular reflection. In regular reflection, the Fresnel equations describe the physics, which includes both reflection and refraction, at the optical boundary of a plate. A "pile of plates" is still a term of art used to describe a polarizer in which a polarized beam is obtained by tilting a pile of plates at an angle to an unpolarized incident beam. The area of polarization was specifically what interested Stokes in this mathematical problem.

For a sample that consists of n layers, each having its absorption, remission, and transmission (ART) fractions symbolized by {a, r, t } , with a + r + t = 1, one may symbolize the ART fractions for the sample as {Α_n, R_n, T_n} and calculate their values by

${\displaystyle T_{n}={\frac {\Omega -{\frac {1}{\Omega }}}{\Omega \Psi ^{n}-{\frac {1}{\Omega \Psi ^{n}}}}},\qquad }$  ${\displaystyle R_{n}={\frac {\Psi ^{n}-{\frac {1}{\Psi ^{n}}}}{\Omega \Psi ^{n}-{\frac {1}{\Omega \Psi ^{n}}}}},\qquad }$  ${\displaystyle A_{n}=1-T_{n}-R_{n},}$ 

where

${\displaystyle \Omega ={\frac {1+r^{2}-t^{2}+\Delta }{2r}},\qquad }$  ${\displaystyle \Psi ={\frac {1-r^{2}+t^{2}+\Delta }{2t}}}$ 

and

${\displaystyle \Delta ={\sqrt {(1+r+t)(1+r-t)(1-r+t)(1-r-t)}}.}$ 

In 1905, in an article entitled "Radiation through a foggy atmosphere", Arthur Schuster published a solution to the equation of radiative transfer, which describes the propagation of radiation through a medium, affected by absorption, emission, and scattering processes.^[12] His mathematics used a two flux approximation; i.e., all light is assumed to travel with a component either in the same direction as the incident beam, or in the opposite direction. He used the word scattering rather than reflection, and considered scatter to be in all directions. He used the symbols k and s for absorption and isotropic scattering coefficients, and repeatedly refers to radiation entering a "layer", which ranges in size from infinitesimal to infinitely thick. In his treatment, the radiation enters the layers at all possible angles, referred to as "diffuse illumination".

In 1931, Paul Kubelka (with Franz Munk) published "An article on the optics of paint", the contents of which has come to be known as the Kubelka-Munk theory. They used absorption and remission (or back-scatter) constants, noting (as translated by Stephen H. Westin) that "an infinitesimal layer of the coating absorbs and scatters a certain constant portion of all the light passing through it". While symbols and terminology are changed here, it seems clear from their language that the terms in their differential equations stand for absorption and backscatter (remission) fractions. They also noted that the reflectance from an infinite number of these infinitesimal layers is "solely a function of the ratio of the absorption and back-scatter (remission) constants a₀/r₀, but not in any way on the absolute numerical values of these constants". This turns out to be incorrect for layers of finite thickness, and the equation was modified for spectroscopic purposes (below), but Kubelka-Munk theory has found extensive use in coatings.^[13][14]

However, in revised presentations of their mathematical treatment, including that of Kubelka, Kortüm and Hecht (below), the following symbolism became popular, using coefficients rather than fractions:

• ${\displaystyle K}$  is the Absorption Coefficient ≡ the limiting fraction of absorption of light energy per unit thickness, as thickness becomes very small.
• ${\displaystyle S}$  is the Back-Scattering Coefficient ≡ the limiting fraction of light energy scattered backwards per unit thickness as thickness tends to zero.

The Kubelka–Munk equation describes the remission from a sample composed of an infinite number of infinitesimal layers, each having a₀ as an absorption fraction, and r₀ as a remission fraction.

${\displaystyle R_{\infty }=1+{\frac {a_{0}}{r_{0}}}-{\sqrt {{\frac {a_{0}^{2}}{r_{0}^{2}}}+2{\frac {a_{0}}{r_{0}}}}}}$ 

Deane Judd was very interested the effect of light polarization and degree of diffusion on the appearance of objects. He made important contributions to the fields of colorimetry, color discrimination, color order, and color vision. Judd defined the scattering power for a sample as Sd, where d is the particle diameter. This is consistent with the belief that the scattering from a single particle is conceptually more important than the derived coefficients.

The above Kubelka–Munk equation can be resolved for the ratio a₀/r₀ in terms of R_∞. This led to a very early (perhaps the first) use of the term "remission" in place of "reflectance" when Judd defined a "remission function" as ${\displaystyle {\frac {(1-R_{\infty })^{2}}{2R_{\infty }}}={\frac {k}{s}}}$ , where k and s are absorption and scattering coefficients, which replace a₀ and r₀ in the Kubelka–Munk equation above. Judd tabulated the remission function as a function of percent reflectance from an infinitely thick sample.^[15] This function, when used as a measure of absorption, was sometimes referred to as "pseudo-absorbance", a term which has been used later with other definitions^[16] as well.

In the 1920s and 30s, Albert H. Taylor, Arthur C. Hardy, and others of the General Electric company developed a series of instruments that were capable of easily recording spectral data "in reflection". Their display preference for the data was "% Reflectance". In 1946, Frank Benford^[2] published a series of parametric equations that gave results equivalent to the Stokes formulas. The formulas used fractions to express reflectance and transmittance.

If A₁, R₁, and T₁ are known for the representative layer of a sample, and A_n, R_n and T_n are known for a layer composed of n representative layers, the ART fractions for a layer with thickness of n + 1 are

${\displaystyle T_{n+1}={\frac {T_{n}T_{1}}{1-R_{n}R_{1}}},\qquad }$  ${\displaystyle R_{n+1}=R_{n}+{\frac {T_{n}^{2}R_{1}}{1-R_{n}R_{1}}},\qquad }$  ${\displaystyle A_{n+1}=1-T_{n+1}-R_{n+1}}$ 

If A_d, R_d and T_d are known for a layer with thickness d, the ART fractions for a layer with thickness of d/2 are

${\displaystyle R_{d/2}={\frac {R_{d}}{1+T_{d}}},\qquad }$  ${\displaystyle T_{d/2}={\sqrt {T_{d}(1-R_{d/2}^{2})}},\qquad }$  ${\displaystyle A_{d/2}=1-T_{d/2}-R_{d/2},}$ 

and the fractions for a layer with thickness of 2d are

${\displaystyle T_{2d}={\frac {T_{d}^{2}}{1-R_{d}^{2}}},\qquad }$  ${\displaystyle R_{2d}=R_{d}(1+T_{2d}),\qquad }$  ${\displaystyle A_{2d}=1-T_{2d}-R_{2d}}$ 

If A_x, R_x and T_x are known for layer x and A_y R_y and T_y are known for layer y, the ART fractions for a sample composed of layer x and layer y are

${\displaystyle T_{x+y}={\frac {T_{x}T_{y}}{1-R_{(-x)}R_{y}}},\qquad }$  ${\displaystyle R_{x+y}=R_{x}+{\frac {T_{x}^{2}R_{y}}{1-R_{(-x)}R_{y}}},\qquad }$  ${\displaystyle A_{x+y}=1-T_{x+y}-R_{x+y}}$ 

The symbol ${\displaystyle R_{(-x)}}$  refers to the reflectance of layer ${\displaystyle x}$  when the direction of illumination is antiparallel to that of the incident beam. The difference in direction is important when dealing with inhomogeneous layers. This consideration was added by Paul Kubelka^[17] in 1954.

In 1955, Ron Giovanelli published explicit expressions for several cases of interest which are touted as exact solutions to the radiative transfer equation for a semi-infinite ideal diffuser.^[9] His solutions have become the standard against which results from approximate theoretical treatments are measured. Many of the solutions appear deceptively simple due to the work of Subrahmanyan (Chandra) Chandrasekhar. For example, the total reflectance for light incident in the direction μ₀ is ${\displaystyle R(\mu _{0})=1-H(\mu _{0}){\sqrt {1-\omega _{0}}}}$ 

Here ω₀ is known as the albedo of single scatter σ/(α+σ), representing the fraction of the radiation lost by scattering in a medium where both absorption (α) and scattering (σ) take place. The function H(μ₀) is called the H-integral, the values of which were tabulated by Chandrasekhar.^[18]

Kortüm was a physical chemist who had a broad range of interests, and published prolifically. His research covered many aspects of light scattering. He began to pull together what was known in various fields into an understanding of how “reflectance spectroscopy” worked. In 1969, the English translation of his book entitled Reflectance Spectroscopy (long in preparation and translation) was published. This book came to dominate thinking of the day for 20 years in the emerging fields of both DRIFTS and NIR Spectroscopy.

Kortüm's position was that since regular (or specular) reflection is governed by different laws than diffuse reflection, they should therefore be accorded different mathematical treatments. He developed an approach based on Schuster's work by ignoring the emissivity of the clouds in the "foggy atmosphere". If we take α as the fraction of incident light absorbed and σ as the fraction scattered isotropically by a single particle (referred to by Kortüm as the "true coefficients of single scatter"), and define the absorption and isotropic scattering for a layer as ${\displaystyle k={\frac {2\alpha }{\alpha +\sigma }}}$  and ${\displaystyle s={\frac {\sigma }{\alpha +\sigma }}}$  then: ${\displaystyle {\frac {(1-R_{\infty })^{2}}{2R_{\infty }}}={\frac {k}{s}}}$ 

This is the same "remission function" as used by Judd, but Kortüm's translator referred to it as "the so-called reflectance function". If we substitute back for the particle properties, we obtain ${\displaystyle {\frac {k}{s}}={\frac {\left({\frac {2\alpha }{\alpha +\sigma }}\right)}{\left({\frac {\sigma }{\alpha +\sigma }}\right)}}=2{\frac {\alpha }{\sigma }}}$  and then we obtain the Schuster equation for isotropic scattering:

${\displaystyle F(R_{\infty })={\frac {(1-R_{\infty })^{2}}{2R_{\infty }}}=2{\frac {\alpha }{\sigma }}}$ 

Additionally, Kortüm derived "the Kubelka-Munk exponential solution" by defining k and s as the absorption and scattering coefficient per centimeter of the material and substituting: K ≡ 2k and S ≡ 2s, while pointing out in a footnote that S is a back-scattering coefficient. He wound up with what he called the "Kubelka–Munk function", commonly called the Kubelka–Munk equation:

${\displaystyle F(R_{\infty })\equiv {\frac {(1-R_{\infty })^{2}}{2R_{\infty }}}={\frac {K}{S}}}$ 

Kortüm concluded that "the two constant theory of Kubelka and Munk leads to conclusions accessible to experimental test. In practice these are found to be at least qualitatively confirmed, and suitable conditions fulfilling the assumptions made, quantitatively as well."

Kortüm tended to eschew the "particle theories", though he did record that one author, N.T. Melamed of Westinghouse Research Labs, "abandoned the idea of plane parallel layers and substituted them with a statistical summation over individual particles."^[19]

In 1966, Harry G. Hecht (with Wesley W. Wendlandt) published a book entitled "Reflectance Spectroscopy", because "unlike transmittance spectroscopy, there were no reference books written on the subject" of "diffuse reflectance spectroscopy", and "the fundamentals were only to be found in the old literature, some of which was not readily accessible".^[20] Hecht describes himself as a novice in the field at the time, and said that if he had known that Gustav Kortüm, "a great pillar in the field", was in the process of writing a book on the subject, he "would not have undertaken the task".^[21] Hecht was asked to write a review of Kortüm's book^[8] and their correspondence concerning it led to Hecht spending a year in Kortüm's laboratories. Kortüm is the author most often cited in the book.

One of the features of the remission function emphasized by Hecht was the fact that

${\displaystyle \log F(R_{\infty })=\log k-\log s}$ 

should yield the absorption spectrum displaced by -log s. While the scattering coefficient might change with particle size, the absorption coefficient, which should be proportional to concentration of an absorber, would be obtainable by a background correction for a spectrum. However, experimental data showed the relationship did not hold in strongly absorbing materials. Many papers were published with various explanations for this failure of the Kubelka-Munk equation. Proposed culprits included: incomplete diffusion, anisotropic scatter ("the invalid assumption that radiation is returned equally in all directions from a given particle"), and presence of regular reflection. The situation resulted in a myriad of models and theories being proposed to correct these supposed deficiencies. The various alternative theories were evaluated and compared.^[3][22]

In his book, Hecht reported the mathematics of Stokes and Melamed formulas (which he called “statistical methods”). He believed the approach of Melamed,^[19] which “involve a summation over individual particles” was more satisfactory than summations over “plane parallel layers”. Unfortunately, Melamed's method failed as the refractive index of the particles approached unity, but he did call attention to the importance of using individual particle properties, as opposed to coefficients that represent averaged properties for a sample. E.L. Simmons used a simplified modification of the particle model to relate diffuse reflectance to fundamental optical constants without the use of the cumbersome equations. In 1975, Simmons evaluated various theories of diffuse reflectance spectroscopy and concluded that a modified particle model theory is probably the most nearly correct.

In 1976, Hecht wrote a lengthy paper comprehensively describing the myriad of mathematical treatments that had been proposed to deal with diffuse reflectance. In this paper, Hecht states that he assumed (as did Simmons) that in the plane-parallel treatment, the layers could not be made infinitesimally small, but should be restricted to layers of finite thickness interpreted as the mean particle diameter of the sample. This is also supported by the observation that the ratio of the Kubelka–Munk absorption and scattering coefficients is 3⁄8 that of corresponding ratio of the Mie coefficients for a sphere. That factor can be rationalized by simple geometric considerations,^[5] recognizing that to a first approximation, the absorption is proportional to volume and the scatter is proportional to cross sectional surface area. This is entirely consistent with the Mie coefficients measuring absorption and scatter at a point, and the Kubelka–Munk coefficients measuring scatter by a sphere.

To correct this deficiency of the Kubelka–Munk approach, for the case of an infinitely thick sample, Hecht blended the particle and layer methods by replacing the differential equations in the Kubelka–Munk treatment by finite difference equations, and obtained the Hecht finite difference formula:

${\displaystyle F(R_{\infty })=a\left({\frac {1}{r}}-1\right)-{\frac {a^{2}}{2r}}}$ 

Hecht apparently did not know that this result could be generalized, but he realized that the above formula "represents an improvement … and shows the need to consider the particulate nature of scattering media in developing a more precise theory".^[3]

Karl Norris pioneered the field of near-infrared spectroscopy.^[23] He began by using log(1/R) as a metric of absorption. While often the samples examined were “infinitely thick”, partially transparent samples were analyzed (especially later) in cells that had a rear reflecting surface (reflector) in a mode called "transflectance". Therefore, the remission from the sample contained light that was back-scattered from the sample, as well as light that was transmitted through the sample, then reflected back to be transmitted through the sample again, thereby doubling the path length. Having no sound theoretical basis for data treatment, Norris used the same electronic processing that was used for absorption data collected in transmission.^[24] He pioneered the use of multiple linear regression for analysis of data.

Gerry Birth was the founder of the International Diffuse Reflectance Conference (IDRC). He also worked at the USDA. He was known to have a deep desire to have a better understanding of the process of light scattering. He teamed up with Harry Hecht (who was active in the early meetings of IDRC) to write the Physics theory chapter, with many photographic illustrations, in an influential Handbook edited by Phil Williams and Karl Norris:^[25] Nearinfrared Technology in the Agriculture and Food Industries.

In 1994, Donald and Kevin Dahm began using numerical techniques to calculate remission and transmission from samples of varying numbers of plane parallel layers from absorption and remission fractions for a single layer. Their plan was to "start with a simple model, treat the problem numerically rather than analytically, then look for analytical functions that describe the numerical results. Assuming success with that, the model would be made more complex, allowing more complex analytical expressions to be derived, eventually, leading to an understanding of diffuse reflection at a level that appropriately approximated particulate samples."^[21] They were able to show the fraction of incident light remitted, R, and transmitted, T, by a sample composed of layers, each absorbing a fraction ${\displaystyle a}$  and remitting a fraction ${\displaystyle r}$  of the light incident upon it, could be quantified by an Absorption/Remission function (symbolized A(R,T) and called the ART function), which is constant for a sample composed of any number of identical layers.

${\displaystyle A(R,T)\equiv {\frac {(1-R_{n})^{2}-T_{n}^{2}}{R_{n}}}={\frac {(2-a-2r)a}{r}}={\frac {a(1+t-r)}{r}}.}$ 

Also from this process came results for several special cases of two stream solutions for plane parallel layers.

For the case of zero absorption, ${\displaystyle R_{n}={\frac {nr}{nr+t}},\qquad }$  ${\displaystyle T_{n}={\frac {t}{nr+t}},}$  ${\displaystyle R_{n}+T_{n}=1}$ .

For the case of infinitesimal layers, ${\displaystyle A(R_{\infty },0)={\frac {(2-a-2r)a}{r}}\approx 2{\frac {a}{r}}=2F(R_{\infty })}$ , and the ART function gives results approaching equivalence to the remission function.

As the void fraction v₀ of a layer becomes large, ${\displaystyle \lim _{v_{0}\to 1}A(R,T)={\frac {(2-\alpha -2\beta )\alpha }{\beta }}\approx 2{\frac {\alpha }{\beta }}}$ .

The ART is related to the Kortüm–Schuster equation for isotopic scatter by ${\displaystyle \lim _{v_{0}\to 1}A(R,T)=4{\frac {\alpha }{\sigma }}}$ .

The Dahms argued that the conventional absorption and scattering coefficients, as well as the differential equations which employ them, implicitly assume that a sample is homogenous at the molecular level. While this is a good approximation for absorption, as the domain of absorption is molecular, the domain of scattering is the particle as a whole. Any approach using continuous mathematics will therefore tend to fail as particles become large.^[26]

Successful application of theory to a real world sample using the mathematics of plane parallel layers requires assigning properties to the layers that are representative of the sample as a whole (which does not require extensively reworking the mathematics). Such a layer was termed a representative layer, and the theory was termed the representative layer theory.^[4]

Furthermore, they argued that it was irrelevant whether the light moving from one layer to another was reflected specularly or diffusely. The reflection and back scatter is lumped together as remission. All light leaving the sample on the same side as the incident beam is termed remission, whether it arises from reflection or back scatter. All light leaving the sample on the opposite side from the incident beam is termed transmission. (In a three-flux or higher treatment, such as Giovanelli's, the forward scatter is not indistinguishable from the directly transmitted light. Additionally, Giovanelli's treatment makes the implied assumption of infinitesimal particles.)

They developed a scheme, subject to the limitations of a two-flux model, to calculate the "scatter corrected absorbance" for a sample.^[27] The decadic absorbance of a scattering sample is defined as −log₁₀(R+T) or −log₁₀(1−A). For a non scattering sample, R = 0, and the expression becomes −log₁₀T or log(⁠1/T⁠), which is more familiar. In a non-scattering sample, the absorbance has the property that the numerical value is proportional to sample thickness. Consequently, a scatter-corrected absorbance might reasonably be defined as one that has that property.

If one has measured the remission and transmission fractions for a sample, R_s and T_s, then the scatter-corrected absorbance should have half the value for half the sample thickness. By calculating the values for R and T for successively thinner samples (s, ⁠1/2⁠s, ⁠1/4⁠s, …) using the Benford's equations for half thickness, a place will be reached where, for successive values of n (0,1,2,3,...), the expression 2ⁿ (−log(R+T)) becomes constant to within a some specified limit, typically 0.01 absorbance units. This value is the scatter-corrected absorbance.

In spectroscopy, remission refers to the reflection or back-scattering of light by a material. While seeming similar to the word "re-emission", it is the light which is scattered back from a material, as opposed to that which is "transmitted" through the material. The word "re-emission" connotes no such directional character. Based on the origin of the word "emit", which means "to send out or away", "re-emit" means "to send out again", "transmit" means "to send across or through", and "remit" means "to send back".

In spectroscopy, the term "plane parallel layers" may be employed as a mathematical construct in discussing theory. The layers are considered to be semi-infinite. (In mathematics, semi-infinite objects are objects which are infinite or unbounded in some, but not all, possible ways.) Generally, a semi-infinite layer is envisioned as a being bounded by two flat parallel planes, each extending indefinitely, and normal (perpendicular) to the direction of a collimated (or directed) incident beam. The planes are not necessarily physical surfaces which refract and reflect light, but may just describe a mathematical plane, suspended in space. When the plane parallel layers have surfaces, they have been variously called plates, sheets, or slabs.

The term "representative layer" refers to a hypothetical plane parallel layer that has properties relevant to absorption spectroscopy that are representative of a sample as a whole. For particulate samples, a layer is representative if each type of particle in the sample makes up the same fraction of volume and surface area in the layer as in the sample. The void fraction in the layer is also the same as in the sample. Implicit in the representative layer theory is that absorption occurs at the molecular level, but that scatter is from a whole particle.

Note: Where a given letter is used in both capital and lower case form (r, R and t ,T ) the capital letter refers to the macroscopic observable and the lower case letter to the corresponding variable for an individual particle or layer of the material. Greek symbols are used for properties of a single particle.

a – absorption fraction of a single layer

r – remission fraction of a single layer

t – transmission fraction of a single layer

A_n, R_n, T_n – the absorption, remission, and transmission fractions for a sample composed of n layers

α – absorption fraction of a particle

β – back-scattering from a particle

σ – isotropic scattering from a particle

k – absorption coefficient, defined as the fraction of incident light absorbed by a very thin layer divided by the thickness of that layer

s – scattering coefficient, defined as the fraction of incident light scattered by a very thin layer divided by the thickness of that layer