Landsat 8 and 9 Underfly International Surface Reflectance Validation Collaboration
by
, , , , , , , , and
Joshua Mann
1,*
Emily Maddox
1
Mahesh Shrestha
1
Jeffrey Irwin
2
Jeffrey Czapla-Myers
3
Aaron Gerace
4
Eon Rehman
4,
Nina Raqueno
4,
Craig Coburn
5
Guy Byrne
6,7,8,9,10,11
Mark Broomhall
6,7,8,9,10,11
Andrew Walsh
6,7,8,9,10,111
KBR Contractor to U.S. Geological Survey Earth Resources Observation and Science Center, Sioux Falls, SD 57198, USA
2
U.S. Geological Survey Earth Resources Observation and Science Center, Sioux Falls, SD 57198, USA
3
Remote Sensing Group, Wyant College of Optical Sciences, University of Arizona, 1630 E University Blvd, Tucson, AZ 85721, USA
4
Digital Imaging and Remote Sensing Laboratory (DIRS), Rochester Institute of Technology, 54 Lomb Memorial Drive, Rochester, NY 14623, USA
5
Department of Geography and Environment, Faculty of Arts and Science, University of Lethbridge, Lethbridge, AB T1K 3M4, Canada
6
Geoscience Australia, Canberra, ACT 2601, Australia
7
School of Earth Sciences, University of Western Australia, Perth, WA 6009, Australia
8
Frontier SI, Docklands, VIC 3008, Australia
9
CSIRO Oceans and Atmosphere, Townsville, QLD 4811, Australia
10
Queensland Department of Environment and Science, Manly, QLD 4179, Australia
11
School of the Environment, University of Queensland, Brisbane, QLD 4072, Australia
*
Author to whom correspondence should be addressed.
Remote Sens. 2024, 16(9), 1492; https://doi.org/10.3390/rs16091492
Submission received: 6 February 2024
/
Revised: 4 April 2024
/
Accepted: 9 April 2024
/
Published: 23 April 2024
(This article belongs to the Special Issue Landsat 9 Pre-launch, Commissioning, and Early On-Orbit Imaging Performance)
Download keyboard_arrow_down
Figure 1
<p>The field site locations are identified by the cyan dots within North America and Australia.</p> "> Figure 2
<p>A brief time series of the cloud conditions of the Guymon, OK, on 14 November 2021 is shown. The field collection began at approximately 16:25 UTC (<b>top</b>). The <b>middle</b> photo was taken toward the middle of the field collection period. The field collection ended at approximately 17:40 UTC (<b>bottom</b>).</p> "> Figure 3
<p>Photos of the Coconino National Forest, AZ, sky conditions are shown. Thin cirrus clouds continued to develop near the field site throughout the field collection period. The field collection began at approximately 17:40 UTC (<b>top</b>) and ended at approximately 18:20 UTC (<b>bottom</b>).</p> "> Figure 4
<p>An overview image of the Jockey Ridge, NC, site taken by the high spatial resolution RGB camera sensors onboard the sUAV platform.</p> "> Figure 5
<p>L9 OLI image of the Ivanpah Playa site (indicated with the blue dot) on 11 November 2021.</p> "> Figure 6
<p>Overview image of the Baldivis, WA, field site.</p> "> Figure 7
<p>An image collected the sUAV of Wilcannia 1 on 15 November is shown.</p> "> Figure 8
<p>Two examples of the land cover at Wilcannia are provided.</p> "> Figure 9
<p>An image of the Southern Alberta field site, collected on by the University of Lethbridge, is shown.</p> "> Figure 10
<p>One-to-one plots, with standard deviation bars, are shown for each of the Landsat 8 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.</p> "> Figure 11
<p>One-to-one plots are shown for each of the Landsat 9 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.</p> ">
Versions Notes
<p>The field site locations are identified by the cyan dots within North America and Australia.</p> "> Figure 2
<p>A brief time series of the cloud conditions of the Guymon, OK, on 14 November 2021 is shown. The field collection began at approximately 16:25 UTC (<b>top</b>). The <b>middle</b> photo was taken toward the middle of the field collection period. The field collection ended at approximately 17:40 UTC (<b>bottom</b>).</p> "> Figure 3
<p>Photos of the Coconino National Forest, AZ, sky conditions are shown. Thin cirrus clouds continued to develop near the field site throughout the field collection period. The field collection began at approximately 17:40 UTC (<b>top</b>) and ended at approximately 18:20 UTC (<b>bottom</b>).</p> "> Figure 4
<p>An overview image of the Jockey Ridge, NC, site taken by the high spatial resolution RGB camera sensors onboard the sUAV platform.</p> "> Figure 5
<p>L9 OLI image of the Ivanpah Playa site (indicated with the blue dot) on 11 November 2021.</p> "> Figure 6
<p>Overview image of the Baldivis, WA, field site.</p> "> Figure 7
<p>An image collected the sUAV of Wilcannia 1 on 15 November is shown.</p> "> Figure 8
<p>Two examples of the land cover at Wilcannia are provided.</p> "> Figure 9
<p>An image of the Southern Alberta field site, collected on by the University of Lethbridge, is shown.</p> "> Figure 10
<p>One-to-one plots, with standard deviation bars, are shown for each of the Landsat 8 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.</p> "> Figure 11
<p>One-to-one plots are shown for each of the Landsat 9 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.</p> ">
Abstract
:During the launch and path to its final orbit, the Landsat 9 satellite performed a once in a mission lifetime maneuver as it passed beneath Landsat 8, resulting in near coincident data collection. This maneuver provided ground validation teams from across the globe the opportunity of collecting surface in situ data to compare directly to Landsat 8 and Landsat 9 data. Ground validation teams identified surface targets that would yield reflectance and/or thermal values that could be used in Landsat Level 2 product validation and set out to collect at these locations using surface validation methodologies the teams developed. The values were collected from each team and compared directly with each other across each of the different bands of both Landsat 8 and 9. The results proved consistency across the Landsat 8 and 9 platforms and also agreed well in surface reflectance underestimation of the Coastal Aerosol, Blue, and SWIR2 bands.
Keywords:
landsat; OLI; TIRS; thermal; multispectral; surface reflectance; calibration; validation; field collection; spectrometer1. Introduction
The Landsat program continued its Earth observation record with the launch of Landsat 9 on 27 September 2021 from Vandenberg Air Force Base, California. Following the launch was a multi-month commissioning phase where the instruments were evaluated prior to approval for standard operations [1]. During the commissioning phase, as Landsat 9 was transitioning to its final orbit, Landsat 9 passed beneath Landsat 8, which is known as the underfly. The underfly provided a unique, once in a mission lifetime opportunity for cross-calibration and validation of Landsat 8 and 9 as both would be observing the same location on Earth within minutes of one another. Previous Landsat underfly missions had been utilized for cross-calibration [2,3]; however, in situ surface reflectance validation efforts were limited. The underfly addressed many calibration and validation challenges present during standard operations when the offset between Landsat 8 and 9 combined is 8 days. Acquisitions being just minutes apart provided minimal changes in atmospheric composition and BRDF effects when both satellites were looking nadir. In addition, Landsat 8 and 9 contain nearly identical instrumentation since the satellites were developed at the same time [4,5].
Given the importance of the underfly to cross-calibration and validation, detailed planning was required to ensure that all goals of the narrow time period were incorporated into the underfly imagery [1]. The underfly was set to occur over an approximate five day period centered on 47.5 days after launch—12 November through 17 November 2021. However, the dates of the underfly could fluctuate as they were completely reliant on the day of launch for Landsat 9. The paths of the underfly were strictly identified as being centered at 47.5 days after the launch, so any delay in launch changed the locations of the underfly path. Surface reflectance validation teams throughout the world worked to utilize the opportunity for validation of the Landsat 8 and 9 derived surface reflectance product. International efforts ultimately resulted in the collection of in situ data from nine different sites scattered across three different countries. Most of the surface reflectance measurements were collected with handheld spectrometers, and the most frequently used was the ASD Fieldspec. In addition to being centered at 47.5 days post-launch, the overlapping coverage of Landsat 8 and 9 transitioned from 0% to 100%—0%, requiring validation teams to be sensitive to the overlapping regions when planning their field collections. Many field teams identified several possible locations for field collection as the final paths collected during the underfly were dependent on the date of the Landsat 9 launch. Once Landsat 9 was officially launched, field teams quickly moved to make final preparations for their fieldwork. The international agencies involved, their field site locations, collection methodology, and instrumentation will be discussed further in Section 2. Results using the data collected by all agencies will be provided in Section 3 and some conclusions in Section 4.
2. Underfly Participants
Months before launch, the approval for the underfly was confirmed, but the dates were uncertain as they were dictated by the launch date of Landsat 9. Surface validation field teams throughout the world began discussions on the collection opportunities made available by the underfly. Unfortunately, the uncertainty of the underfly paths and dates made it difficult for some teams to identify a field site for collection. In addition, the event was confirmed only a few months notice prior to the underfly which limited some agencies’ ability to obtain the instrumentation needed for the fieldwork. The global COVID-19 pandemic created additional complexity to travel, site access, and equipment access for many groups. Finally, weather conditions over the field sites were limiting factors for some groups. Five agencies and universities were able to participate in field measurements with little interference from clouds throughout the underfly period. These five groups included USGS ECCOE, University of Arizona, Rochester Institute of Technology (Rochester, NY, USA), Geoscience Australia, and University of Lethbridge (Lethbridge, AB, Canada). Field collection was performed at nine different field sites within the U.S., Canada, and Australia, shown in Figure 1.
2.1. EROS Calibration and Validation Center of Excellence (ECCOE)
The ECCOE field team focused their potential field sites in the South and Southwestern areas of the United States. These areas were chosen to provide the highest likelihood of cloud-free field collection opportunities. Guymon, Oklahoma, USA, and Coconino National Forest, Arizona, USA, were the final sites chosen on the underfly paths for surface reflectance validation. The two field sites were approximately 1125 km (700 miles) apart and data were collected by the field team on 14 November and 15 November, respectively.
The Guymon field site was located on Oklahoma state-leased land, composed of thick grasses and cacti. The grasses were approximately 20 cm in height, shown in the middle photo of Figure 2. The site measured 200 m by 200 m with increasing elevation on the north end of the site. The cloud cover increased significantly throughout the field collection period as cirrocumulus clouds developed, as seen in Figure 2, impacting measurements.
The Coconino National Forest field site was located approximately 29 km (18 miles) north of Flagstaff, AZ, composed of dry grass and cacti, visible in Figure 3. The site measured 200 m by 200 m and remained at a consistent elevation. Thin cirrus clouds were present at the beginning of the field collection, spreading across the sky throughout the field collection period as can be seen in Figure 3.
ECCOE utilized their recently developed dual spectroradiometer approach [6] at both field sites. Their methodology included two ASD FieldSpec 4 spectroradiometers. The ASD FieldSpec 4 is a full spectrum spectroradiometer measuring 350–2500 nm. An 8° foreoptic was used on the ASD and 30 cm (12 inch) 99% reflectance white panels were used as a calibration source for surface reflectance measurements. The dual system methodology consisted of having one ASD taking reflectance panel measurements every 2–3 s throughout the entire field collection period. A second ASD was carried by a team member, taking measurements as the team member walked along each transect. More information about the dual-ASD system methodology used by ECCOE can be found in Shrestha et al. 2023 [6].
2.2. Rochester Institute of Technology
The RIT collected surface reflectance measurements at Jockey’s Ridge State Park, North Carolina, USA. Jockey’s Ridge is the location of the “tallest living sand dune on the Atlantic coast” [7], providing a thermally stable, large, homogeneous area for observations, shown in Figure 4. RIT completed collection of both surface reflectance and surface temperature; although, only surface reflectance is discussed in this paper. Additional information on their surface temperature validation work can be found in Eon et al. 2023 [8].
RIT utilized a UAV-based instrumentation approach. Surface reflectance measurements were made by two Headwall sensors—the Headwall Nano and Headwall SWIR M384. Together, they cover 400–2500 nm in 2.2 nm and 10 nm increments, respectively [9]. Image and GPS/IMU data were collected by the MX-1 payload which was used to georectify the data. A high spatial resolution RGB camera was flown on a DJI Mavic to produce a basemap of the field site.
2.3. University of Arizona
The UArizona field team travelled to the Ivanpah Playa on the California–Nevada border, USA, a CEOS LandNet site [10,11] shown in Figure 5. The field site was a large, homogeneous dry lake-bed within the Mohave Desert. This site is well suited and frequently used for calibration across several remote sensing optical missions [12,13,14]. Conditions remained cloud-free throughout the field collection period. UArizona used a single ASD FieldSpec 4, as discussed in Section 2.1, throughout their field collection, including an 8° foreoptic, and a 30 cm (12 inch) 99% reflectance white panel was used as a reference.
2.4. Geoscience Australia
Geoscience Australia worked in collaboration with Environment NSW, University of Western Australia, Frontier SI, CSIRO Oceans and Atmosphere, Queensland Department of Environment and Science, and University of Queensland. The collaborative group, referred to here as Geoscience Australia, identified several sites throughout Australia during the entire underfly period to maximize the opportunities for successful collection. Four of the surface reflectance field collects from Geoscience Australia are discussed here—Baldivis, Wilcannia 1, Wilcannia 2, and Narromine. In addition to multiple contributors, Landsat data used within the Geoscience Australia analysis were BRDF corrected [15,16,17].
Baldivis, located in Western Australia, was a flat, grassy site in a rural area Northeast of the town of Baldivis, shown in Figure 6. The field site was approximately 90 m by 90 m. Field collection was conduced on 13 November 2021 with ∼15% overlap between Landsat 8 and 9. A single ASD, as described in Section 2.1, was used for field collection.
Field collection on 15 November 2021 was conducted over two field sites east of Wilcannia, New South Wales. The first field site, denoted here as Wilcannia 1 shown in Figure 7, was composed of patchy vegetation and bare ground, measuring approximately 100 m by 100 m. Field measurements at Wilcannia 1 were focused on manual collection using an ASD. The second field site, denoted here as Wilcannia 2, was also composed of patchy vegetation and bare ground similar to what is shown in Figure 8. Measurement methodologies were focused on both manual and UAV platforms. Manual collection, using a SR3500, was conducted over an area of approximately 100 m by 100 m. The SR3500 is a portable full range spectroradiometer manufactured by Spectral Evolution in Haverhill, MA that measures 350–2500 nm. For the UAV collection, the FLAME instrument, a miniature spectroradiometer measuring 190–1100 nm, was used over an area of 150 m by 125 m. A majority of the collection areas within Wilcannia 2 were overlapping. Although clouds were initially forecast for the Landsat overpass times, skies remained clear for field collection.
Lastly, Narromine, located in New South Wales, was a flat area with patchy vegetation near the end of a runway at the Narromine Airport. The collection area was approximately 115 m by 115 m, collected on 17 November 2021. Both an ASD and SR3500 were used for surface reflectance validation. Narromine is the only field site observed during the off nadir period of the underfly, 17 November, which was completed to increase the overlap between Landsat 8 and 9.
2.5. University of Lethbridge, Canada
The University of Lethbridge collected at one field site in Southern Alberta. The field site was 200 m by 200 m, composed of dry grazed native grassland, shown in Figure 9. Sky conditions remained free of clouds over the field site throughout the approximately 45 min collection period. University of Lethbridge instrumentation included an ASD FieldSpec 3 High Resolution spectroradiometer, a full range spectroradiometer measuring 350–2500 nm, using an 8° foreoptic. A standard 30 cm (12 inch) square Spectralon panel was used as the white reference panel. White panel measurements were taken at the eastern edge of each transect walked.
2.6. Additional Interested Agencies
ECCOE validation field team members met with international agencies once the underfly was approved for planning. Many Landsat International Cooperators were interested in the field collection opportunity; however, most were unable to participate. Due to the short period of time for planning after the final underfly paths were announced and weather impacts during the underfly period, some field collection activities were unable to be completed. NRSC/ISRO had planned to collect at Shadnagar, Telengana, India, but were unable to complete the collection due to weather. Similarly, LAPAN had planned to collect at Punjullharjo-Rembang, Central Java, Indonesia, but weather conditions interfered with measurements. Several other Landsat International Cooperators expressed interest, including SANSA, KSAT, AGEOS, and AIR. In addition, DEMMIN also participated in underfly field collection discussions.
3. Results
Field collection surface reflectance results at nine field sites are compared to Landsat 8 and Landsat 9 derived surface reflectance values. Field collection data are spatially positioned using their GPS measurements, stacked by spectra, then averaged throughout the field site. The reflectance values are adjusted to the Landsat bands by using the L8 and L9 relative spectral responses. Next, the Landsat pixels that fell within each site’s region of interest were identified, and the pixel values were averaged together and compared to the averaged field collection reflectance data.
The Guymon field collection brings forward the impact of clouds within surface reflectance validation. In a traditional field collection setting, field teams typically do not go out to collect measurements if clouds are likely to form throughout the collection period. However, during an underfly, field teams have little choice to collect if clouds may be a possibility throughout the collection period. Throughout the field collection period, cirrocumulus clouds developed over the field site, as can be seen in Figure 2. Given the short time frame in which the clouds developed, the clouds at the Landsat overpass times are not necessarily representative of the average cloud cover observed throughout the entire field collection period. While the clouds impact all bands, the SWIR1 band shows the least unity as the ratios of Landsat 8 to ground truth and Landsat 9 to ground truth were 0.768 and 0.764, respectively. The ratios of the SWIR1 band are some of the furthest from unity throughout all field collection observations during the underfly. As can be seen by the lime green markers in Figure 10 and Figure 11, the offset between the ground truth and each satellite become more defined throughout the longer wavelengths. The absolute difference of Landsat 8 and 9 to ground truth is 0.045 and 0.047 r.u., accordingly reflectance units. However, Landsat 8 and 9 were consistent in their observations as the absolute differences of the Landsat surface reflectance products ranged from 0.000 to 0.002 reflectance units, building confidence in the Landsat measurements.
Coconino National Forest field collection conditions included some light cirrus clouds that spread out throughout the collection period, as can be seen in Figure 3. Ratios of Landsat 8 to ground truth range from 0.928 to 0.975. Shown by the forest green markers in Figure 10 and Figure 11, as well as the ratios being less than 1.0 in Table 1, the results indicate Landsat 8 is underestimating the surface reflectance throughout the field site. Absolute differences of Landsat 8 and ground truth range from 0.003 to 0.011 reflectance units. Similarly, the ratio of Landsat 9 to ground truth that furthest from unity is 0.928 in the Coastal Aerosol band. However, it is important to note the ratio in the SWIR2 band is at unity. Absolute differences between Landsat 9 and ground truth were slightly lower than those with Landsat 8, ranging from 0.000 to 0.009 reflectance units. Again, the ratios seen indicate Landsat 9 is underestimating the surface reflectance in most bands. Lastly, the Landsat 8 and 9 absolute differences are within the expected range of the Landsat product, with differences ranging from 0.000 to 0.004 reflectance units.
Ivanpah Playa is the brightest target observed throughout the underfly field collections, shown in the red markers in Figure 10 and Figure 11. Similarly to Guymon and Coconino, the results indicate that Landsat 8 and 9 are underestimating the surface reflectance in all bands. Looking first at Landsat 8, the SWIR1 band is closest to unity with a ratio of 0.955, and the Coastal Aerosol is furthest from unity with a ratio of 0.911. Differences between the Landsat 8 surface reflectance product and the ground truth range from 0.017 to 0.028 reflectance units, with the SWIR1 band having an absolute difference of 0.011 as expected by the ratio. The ratios of Landsat 9 to ground truth are further from unity when compared to the Landsat 8 ratios, excluding the SWIR bands where the ratio of the SWIR1 band is also 0.955. As expected, differences between Landsat 9 and the ground truth are greater than those seen with Landsat 8 as absolute differences ranged from 0.020 to 0.026. With that said, the Landsat 8 and 9 surface reflectance values are comparable with absolute differences of 0.000 to 0.004 reflectance units.
Jockey’s Ridge is the other bright target observed throughout the underfly. As seen with Guymon, Coconino National Forest, and Ivanpah Playa, the ratios of both Landsat 8 and Landsat 9 to ground truth indicate that the Landsat surface reflectance product is underestimating the surface reflectance. Ratios of Landsat 8 to ground truth range from 0.841 to 0.981. The disagreement between Landsat 8 and the ground truth is most apparent in the SWIR bands with the ratio of SWIR1 being 0.908, and the ratio of SWIR2 being 0.841. The NIR band is closest to unity with a ratio of 0.981. Absolute differences between Landsat 8 and the ground truth are greater than typically expected, ranging from 0.006 to 0.089 reflectance units. As expected with the Landsat product, similar results are seen with Landsat 9. Ratios of Landsat 9 to ground truth range from 0.846 to 0.976, again with the SWIR2 band being furthest from unity and the NIR band closest to unity. Absolute differences between Landsat 9 and ground truth are also similar, varying from 0.006 to 0.086 reflectance units, greater than anticipated with the Landsat surface reflectance products. However, the absolute differences between Landsat 8 and 9 are only 0.000 to 0.003 reflectance units, falling within the expected range of the Landsat product.
The Baldivis field site, composed of patchy green and dry grass, differs from previous results with Landsat 8 as the ratios of Landsat 8 to ground truth are greater than 1 in all bands except for the SWIR2 band, indicating Landsat 8 is overestimating the surface reflectance at the site. Of note is the ratio of the Coastal Aerosol band, 1.221, with the greatest overestimation of Landsat 8 surface reflectance to ground truth throughout all field sites in the underfly. Absolute differences between Landsat 8 and the ground truth range from 0.006 to 0.018 reflectance units. The ratio of Landsat 9 to the ground truth indicates the Landsat product is underestimating the surface reflectance as is seen with other sites in nearly all bands. Again, shown in Table 1, the Coastal Aerosol band has the Landsat 9 to ground truth ratio furthest from unity at 0.867. Absolute differences between Landsat 9 and ground truth range from 0.001 to 0.012. The absolute differences between Landsat 8 and 9 are greater than previous sites as the differences were 0.007 to 0.010.
Handheld spectrometers are very commonly used in manual field collection, with the ASD FieldSpec most frequently used. Data collected by Geoscience Australia during the underfly not only allow an assessment of Landsat data, they allow for a brief look at the similarities and differences of the two handheld spectrometers used—ASD and SR3500. Although, it is important to note that there are not sufficient data within this exercise to conclude the general results observed with each instrument. Given the proximity of Wilcannia 1 and 2, ∼2.8 km, the land cover and atmospheric composition differences are reduced, allowing insight into the two instruments. The ASD and SR3500 results both show an underestimation of Landsat 8 surface reflectance in the Coastal Aerosol and Blue bands, and an overestimation of surface reflectance in the Green through SWIR1 bands. The absolute differences between Landsat 8 and the ASD at Wilcannia 1 range from 0.000 to 0.015 reflectance units, while the differences between Landsat 8 and the SR3500 at Wilcannia 2 range from 0.007 to 0.022 reflectance units. The FLAME instrument onboard a UAV is also used at Wilcannia 2. The ratio of Landsat 8 to the FLAME ground truth is further from unity than expected with a ratio of 0.747 and an absolute difference of 0.038 reflectance units. However, the ground truth in the NIR band is nearly at unity with a ratio of 0.999 and an absolute difference of 0.000 reflectance units. Wilcannia 1 Landsat 9 results largely show an overestimation of Landsat 9 surface reflectance in all bands except the Coastal Aerosol band. The greatest absolute difference between Landsat 9 and ground truth is 0.027 reflectance units in the NIR band, while the lowest absolute difference of 0.004 reflectance units in the Blue band. A similar story is seen again with the SR3500 at Wilcannia 2—ratios show an underestimation of the Landsat 9 surface reflectance product in the Coastal Aerosol and Blue bands, and an overestimation in surface reflectance in all other bands. The Landsat 9 to ground truth ratio furthest from unity is the Coastal Aerosol band, where an absolute difference of 0.022 reflectance units is present. The FLAME results for Landsat 9 are very similar to Landsat 8, where again the NIR band is nearly at unity with a ratio of 1.001. The differences between Landsat 8 and 9 at each location vary more than anticipated given their proximity to one another and similar land cover type. Absolute differences of Landsat 8 and 9 at Wilcannia 1 are 0.006 to 0.012, whereas the differences at Wilcannia 2 only range from 0.000 to 0.003 reflectance units.
Narromine also includes data from two instrument sources—an ASD and SR3500. The ratios of Landsat 8 to the ground truth once again are consistently below 1, where the Landsat surface reflectance product is underestimating the surface reflectance of the field site. Ratios of the ASD range from 0.813 to 0.963, with the Coastal Aerosol band being furthest from unity. Similarly, the Coastal Aerosol band is one of the bands furthest from unity when looking at the SR3500, with a ratio of 0.847. Absolute differences between Landsat 8 and the ASD ground truth range from 0.010 to 0.027 reflectance units. The ranges of absolute difference between Landsat 8 and the SR3500 ground truth are similar, but slightly greater, going from 0.011 to 0.030 reflectance units. Landsat 9 results show that, for the Coastal Aerosol through NIR bands, the Landsat surface reflectance product is overestimating the surface reflectance of the field site. Ratios of Landsat 9 to ASD ground truth range from 0.968 in the SWIR2 band, to 1.088 in the Coastal Aerosol band. Ratios of Landsat 9 to SR3500 ground truth range from 0.979 to 1.070 reflectance units. The absolute differences between Landsat 9 and the ASD ground truth range from 0.002 to 0.009 reflectance units, and the difference between Landsat 9 and the SR3500 ground truth range from 0.002 to 0.008 reflectance units. Lastly, the absolute difference between Landsat 8 and 9 is greater than expected from the product. Differences range from 0.013 in the Blue and Green bands to 0.025 in the SWIR1 band.
As can be seen in Table 1, ground truth results are greater than the Landsat derived surface reflectance in the Coastal Aerosol, Blue, and SWIR2 bands, similar to the ASD and SR3500 results at Wilcannia 1 and 2. The ratio of Landsat 8 to ground truth ranges from 0.858 in the Coastal Aerosol band to 1.041 in the Red band, and the absolute differences range from 0.001 to 0.009 reflectance units. Landsat 9 provides similar results with the ratio of Landsat 9 to ground truth ranging from 0.855 in the Coastal Aerosol band to 1.027 in the Red band. Absolute differences between Landsat 9 and ground truth have a larger, but still acceptable, range of 0.001 to 0.009 reflectance units. Finally, as is seen with previous field sites, the differences between Landsat 8 and 9 range from 0.000 to 0.004, as expected.
4. Conclusions
The Landsat 8 and 9 underfly provided a once in a mission lifetime opportunity for surface reflectance validation, utilizing near-simultaneous satellite acquisitions. To the best of the authors’ knowledge, a large scale manual validation effort has not been conducted with previous underfly opportunities, making it a rare and important campaign to understand Landsat 8 and 9 surface reflectance data quality. Five agencies completed field collection throughout the underfly period—ECCOE, RIT, UArizona, Geoscience Australia (composed of seven agencies and universities), and University of Lethbridge. Many challenges were encountered throughout the preparation period of the underfly as interest in field collection grew. While there were several agencies interested in collecting data, the uncertainty of the paths and limited time between the launch of Landsat 9 (and therefore the path identification) and the underfly period was limiting. In addition, agencies that were new to field collection did not have enough time to acquire the needed instrumentation for field collection. Lastly, weather impacted agencies prepared for field collection—NRSC/ISRO and LAPAN.
Field collection was conducted over nine different field sites throughout North America and Australia, providing measurements for a variety of land cover including dry grass, patchy green and dry grass, patchy green vegetation and bare Earth, desert, and loose sand. Both manual and airborne instrumentation were used throughout the field collection sites. ASDs were used at Guymon, Coconino National Forest, Ivanpah Playa, Baldivis, Wilcannia 1, Narromine, and Southern Alberta. A SR3500 was used at Wilcannia 2 and Narromine. Finally, UAV platforms were used at Jockey Ridge and Wilcannia 2, with Jockey Ridge measuring with a Headwall Nano and SWIR M384 and Wilcannia 2 measuring with a FLAME spectrometer.
The Landsat 8 Coastal Aerosol and Blue bands showed an underestimation of Landsat surface reflectance; Baldivis was the only outlier. In addition, the Landsat 8 SWIR2 band underestimated the Landsat surface reflectance at all locations and instrumentation excluding Wilcannia 2 SR3500. Similar results were seen within the Landsat 9 results where the Coastal Aerosol, Blue, and SWIR2 surface reflectance was underestimated at a majority of sites. The ratio of Landsat 9 to the Coconino National Forest ground truth in SWIR2 was the only field observation that reached unity with a Landsat satellite. Results from the Wilcannia 2 airborne observations using FLAME were nearly at unity with the ratio of Landsat 8 to ground truth at 0.0009, and the ratio of Landsat 9 to ground truth at 1.001.
Throughout standard field collection operations, collecting in the presence of clouds is not typically conducted. However, the underfly opportunity was extremely rare, therefore pushing the field team at Guymon to collect despite cirrocumulus cloud development throughout the collection period. As anticipated, the results were not very close to unity, particularly in the SWIR bands. With that said, it is important to note that one factor from that day was the rapid development of the cirrocumulus clouds. Given field collection could take 40–60 min depending on the field site, the cloud cover at the time of the Landsat overpasses may not have been indicative of the average cloud cover observed throughout the entire field collection period.
While the number of acquisitions was limited during the underfly, the field collection efforts made throughout North America and Australia provided irreplaceable data for such a unique opportunity. Surface reflectance validation has been an area of consistent growth throughout the past decade as more field teams have begun manually taking measurements. The ability to manually take measurements was a critical factor in successful field collection throughout the underfly as the number of paths acquired were limited and could not be guaranteed to acquire data over automated instrumentation sites. Through the underfly process, the importance of planning and communication was a necessity. Organizing field collection locations was challenging in the months leading up to the Landsat 9 launch as the paths of the underfly were dictated by the Landsat 9 launch date, as well as ensuring all field teams were able to travel to their field collection sites. Given the number of variables that impacted each team’s ability to collect field data, the collaborative effort of collecting surface reflectance validation was a success and provides insight into future underfly opportunities with Landsat launches moving forward.
Author Contributions
Conceptualization, J.M.; Methodology, J.M.; Formal analysis, J.M., J.C.-M., M.S., C.C., A.G., M.B., G.B., A.W., E.R. and N.R.; Data curation, J.M., C.C., A.G., M.B., G.B., A.W., N.R. and J.I.; Writing—original draft, E.M. All authors have read and agreed to the published version of the manuscript.
Funding
ECCOE field collection is funded under USGS contract 140G0121D0001.
Data Availability Statement
Data collected by ECCOE are available on ScienceBase at https://doi.org/10.5066/P9E3F6LV (accessed on 25 September 2023). Data collected by Geoscience Australia are available at https://www.dea.ga.gov.au/products/national-spectral-database/ (accessed on 20 November 2021). Data collected by RIT, UArizona, and the University of Lethbridge are not available.
Acknowledgments
The U.S. Geological Survey ECCOE project lead was Cody Anderson, and the U.S. Geological Survey ECCOE field collection team lead was Jeff Irwin. The University of Lethbridge field collection team included Gordon Logie and Caleb Fowler. The Environment NSW team included Tony Gill, Heidi Mawbey, Geof Dale, and Geof Horn. The Geoscience Australia collection team also included Medhavy Thankappan and Fuqin Li.
Conflicts of Interest
The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
Abbreviations
| AGEOS | Gabonese Studies and Space Observations Agency |
| AIR | Aerospace Information Research Institute |
| ASD | Analytical Spectral Device |
| BRDF | Bidirectional Reflectance Distribution Function |
| CEOS | Committee on Earth Observation Satellites |
| CSIRO | Commonwealth Scientific and Industrial Research Organization |
| DEMMIN | Durable Environmental Multidisciplinary Monitoring Information Network |
| ECCOE | EROS Calibration and Validation Center of Excellence |
| GPS | Global Positioning System |
| IMU | Inertial Measurement Unit |
| ISRO | Indian Space Research Organization |
| KSAT | Kongsberg Satellite Services |
| LandNet | Land-equipped Network |
| LAPAN | National Institute of Aeronautics and Space |
| NIR | Near Infrared |
| NRSC | National Remote Sensing Centre |
| NSW | New South Wales |
| RIT | Rochester Institute of Technology |
| SANSA | South African National Space Agency |
| SI | Spatial Information |
| SWIR | Short-Wave Infrared |
| UArizona | University of Arizona |
| USGS | U.S. Geological Survey |
| UAV | Unmanned Aircraft Vehicle |
| WRS-2 | World Reference System Two |
References
- Kaita, E.; Markham, B.; Haque, M.O.; Dichmann, D.; Gerace, A.; Leigh, L.; Good, S.; Schmidt, M.; Crawford, C.J. Landsat 9 Cross Calibration Under-Fly of Landsat 8: Planning, and Execution. Remote Sens. 2022, 14, 5414. [Google Scholar] [CrossRef]
- Teillet, P.; Barker, J.; Markham, B.; Irish, R.; Fedosejevs, G.; Storey, J. Radiometric cross-calibration of the Landsat-7 ETM+ and Landsat-5 TM sensors based on tandem data sets. Remote Sens. Environ. 2001, 78, 39–54. [Google Scholar] [CrossRef]
- Mishra, N.; Haque, M.O.; Leigh, L.; Aaron, D.; Helder, D.; Markham, B. Radiometric Cross Calibration of Landsat 8 Operational Land Imager (OLI) and Landsat 7 Enhanced Thematic Mapper Plus (ETM+). Remote Sens. 2014, 6, 12619–12638. [Google Scholar] [CrossRef]
- Masek, J.G.; Wulder, M.A.; Markham, B.; McCorkel, J.; Crawford, C.J.; Storey, J.; Jenstrom, D.T. Landsat 9: Empowering open science and applications through continuity. Remote Sens. Environ. 2020, 248, 111968. [Google Scholar] [CrossRef]
- Gross, G.; Helder, D.; Begeman, C.; Leigh, L.; Kaewmanee, M.; Shah, R. Initial Cross-Calibration of Landsat 8 and Landsat 9 Using the Simultaneous Underfly Event. Remote Sens. 2022, 14, 2418. [Google Scholar] [CrossRef]
- Shrestha, M.; Mann, J.; Maddox, E.; Robbins, T.; Irwin, J.; Helder, D. Implementing Dual Spectrometer Approach for Improved Surface Reflectance Estimation. Remote Sens. 2023, 15, 5451. [Google Scholar] [CrossRef]
- NC Division of Parks & Recreation. Jockey’s Ridge State Park. Available online: https://www.ncparks.gov/jockeys-ridge-state-park/home (accessed on 22 June 2022).
- Eon, R.; Gerace, A.; Falcon, L.; Poole, E.; Kleynhans, T.; Raqueño, N.; Bauch, T. Validation of Landsat-9 and Landsat-8 Surface Temperature and Reflectance during the Underfly Event. Remote Sens. 2023, 15, 3370. [Google Scholar] [CrossRef]
- Kaputa, D.S.; Bauch, T.; Roberts, C.; McKeown, D.; Foote, M.; Salvaggio, C. MX-1: A New Multi-Modal Remote Sensing UAS Payload with High Accuracy GPS and IMU. In Proceedings of the 2019 IEEE Systems and Technologies for Remote Sensing Applications Through Unmanned Aerial Systems (STRATUS), Rochester, NY, USA, 25–27 February 2019; pp. 1–4. [Google Scholar] [CrossRef]
- CEOS. CEOS Cal/Val Portal, CEOS LandNet: Ivanpah Playa. Available online: https://calvalportal.ceos.org/ivanpah-playa (accessed on 22 June 2022).
- ECCOE. ECCOE Test Sites Catalog: Ivanpah Playa. Available online: https://calval.cr.usgs.gov/apps/ivanpah (accessed on 22 June 2022).
- Mahalingam, S.; Srinivas, P.; Devi, P.K.; Sita, D.; Das, S.K.; Leela, T.S.; Venkataraman, V.R. Reflectance based vicarious calibration of HySIS sensors and spectral stability study over pseudo-invariant sites. In Proceedings of the 2019 IEEE Recent Advances in Geoscience and Remote Sensing: Technologies, Standards and Applications (TENGARSS), Kochi, India, 17–20 October 2019; pp. 132–136. [Google Scholar] [CrossRef]
- Naughton, D.; Brunn, A.; Czapla-Myers, J.S.; Douglass, S.; Thiele, M.; Weichelt, H.; Oxfort, M. Absolute radiometric calibration of the RapidEye multispectral imager using the reflectance-based vicarious calibration method. J. Appl. Remote Sens. 2011, 5, 053544. [Google Scholar] [CrossRef]
- Villa-Aleman, E. Assessment of Ivanpah Playa as a Site for Thermal Vicarious Calibration for the MTI Satellite. In Proceedings of the AEROSENSE 2003, Orlando, FL, USA, 21–25 April 2003; Volume 5093. [Google Scholar] [CrossRef]
- Li, F.; Jupp, D.L.B.; Reddy, S.; Lymburner, L.; Mueller, N.; Tan, P.; Islam, A. An Evaluation of the Use of Atmospheric and BRDF Correction to Standardize Landsat Data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2010, 3, 257–270. [Google Scholar] [CrossRef]
- Li, F.; Jupp, D.L.; Thankappan, M.; Lymburner, L.; Mueller, N.; Lewis, A.; Held, A. A physics-based atmospheric and BRDF correction for Landsat data over mountainous terrain. Remote Sens. Environ. 2012, 124, 756–770. [Google Scholar] [CrossRef]
- Li, F.; Jupp, D.L.; Paget, M.; Briggs, P.R.; Thankappan, M.; Lewis, A.; Held, A. Improving BRDF normalisation for Landsat data using statistical relationships between MODIS BRDF shape and vegetation structure in the Australian continent. Remote Sens. Environ. 2017, 195, 275–296. [Google Scholar] [CrossRef]
Figure 1.
The field site locations are identified by the cyan dots within North America and Australia.
Figure 1.
The field site locations are identified by the cyan dots within North America and Australia.
Figure 2.
A brief time series of the cloud conditions of the Guymon, OK, on 14 November 2021 is shown. The field collection began at approximately 16:25 UTC (top). The middle photo was taken toward the middle of the field collection period. The field collection ended at approximately 17:40 UTC (bottom).
Figure 2.
A brief time series of the cloud conditions of the Guymon, OK, on 14 November 2021 is shown. The field collection began at approximately 16:25 UTC (top). The middle photo was taken toward the middle of the field collection period. The field collection ended at approximately 17:40 UTC (bottom).
Figure 3.
Photos of the Coconino National Forest, AZ, sky conditions are shown. Thin cirrus clouds continued to develop near the field site throughout the field collection period. The field collection began at approximately 17:40 UTC (top) and ended at approximately 18:20 UTC (bottom).
Figure 3.
Photos of the Coconino National Forest, AZ, sky conditions are shown. Thin cirrus clouds continued to develop near the field site throughout the field collection period. The field collection began at approximately 17:40 UTC (top) and ended at approximately 18:20 UTC (bottom).
Figure 4.
An overview image of the Jockey Ridge, NC, site taken by the high spatial resolution RGB camera sensors onboard the sUAV platform.
Figure 4.
An overview image of the Jockey Ridge, NC, site taken by the high spatial resolution RGB camera sensors onboard the sUAV platform.
Figure 5.
L9 OLI image of the Ivanpah Playa site (indicated with the blue dot) on 11 November 2021.
Figure 6.
Overview image of the Baldivis, WA, field site.
Figure 7.
An image collected the sUAV of Wilcannia 1 on 15 November is shown.
Figure 8.
Two examples of the land cover at Wilcannia are provided.
Figure 9.
An image of the Southern Alberta field site, collected on by the University of Lethbridge, is shown.
Figure 9.
An image of the Southern Alberta field site, collected on by the University of Lethbridge, is shown.
Figure 10.
One-to-one plots, with standard deviation bars, are shown for each of the Landsat 8 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.
Figure 10.
One-to-one plots, with standard deviation bars, are shown for each of the Landsat 8 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.
Figure 11.
One-to-one plots are shown for each of the Landsat 9 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.
Figure 11.
One-to-one plots are shown for each of the Landsat 9 bands, including results from each field site and instrument. The Guymon site is represented by lime green, Coconino National Forest represented by green, Ivanpah Playa represented by red, Jockey Ridge represented by orange, Baldivis represented by pink, Narromine sASD represented by navy, Narromine SR3500 represented by sky blue, Wilcannia 1 represented by purple, Wilcannia 2 SR3500 represented by slate blue, Wilcannia 2 FLAME represented by dark turquoise, and Southern Alberta represented by fuchsia.
Table 1.
The ratio of Landsat 8 to ground truth collected by field teams is shown on the top half of the table. The ratio of Landsat 9 to ground truth collected by field teams is shown on the bottom half of the table.
Table 1.
The ratio of Landsat 8 to ground truth collected by field teams is shown on the top half of the table. The ratio of Landsat 9 to ground truth collected by field teams is shown on the bottom half of the table.
| Ratio of Landsat 8 to Ground Truth | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Landsat Band | Guymon (ASD) | Coconino (ASD) | Ivanpah Playa (ASD) | Jockey Ridge (Headwall) | Baldivis (ASD) | Wilcannia 1 (ASD) | Wilcannia 2 (SR3500) | Wilcannia 2 (FLAME) | Narromine (ASD) | Narromine (SR3500) | Southern Alberta (ASD) |
| Coastal Aerosol | 0.900 | 0.928 | 0.911 | 0.957 | 1.221 | 0.878 | 0.835 | 0.747 | 0.813 | 0.847 | 0.858 |
| Blue | 0.923 | 0.951 | 0.928 | 0.964 | 1.180 | 0.965 | 0.946 | 0.868 | 0.861 | 0.847 | 0.980 |
| Green | 0.905 | 0.971 | 0.936 | 0.916 | 1.104 | 1.024 | 1.037 | 0.960 | 0.902 | 0.893 | 1.028 |
| Red | 0.865 | 0.978 | 0.952 | 0.938 | 1.089 | 1.043 | 1.063 | 0.994 | 0.914 | 0.907 | 1.041 |
| NIR | 0.824 | 0.962 | 0.952 | 0.981 | 1.061 | 1.053 | 1.052 | 0.999 | 0.963 | 0.949 | 1.027 |
| SWIR1 | 0.768 | 0.948 | 0.955 | 0.908 | 1.031 | 1.034 | 1.045 | 0.927 | 0.920 | 1.027 | |
| SWIR2 | 0.791 | 0.975 | 0.943 | 0.841 | 0.980 | 0.999 | 1.028 | 0.892 | 0.902 | 0.987 | |
| Ratio of Landsat 9 to Ground Truth | |||||||||||
| Landsat Band | Guymon (ASD) | Coconino (ASD) | Ivanpah Playa (ASD) | Jockey Ridge (Headwall) | Baldivis (ASD) | Wilcannia 1 (ASD) | Wilcannia 2 (SR3500) | Wilcannia 2 (FLAME) | Narromine (ASD) | Narromine (SR3500) | Southern Alberta (ASD) |
| Coastal Aerosol | 0.896 | 0.928 | 0.900 | 0.950 | 0.867 | 0.936 | 0.844 | 0.755 | 1.088 | 1.070 | 0.855 |
| Blue | 0.905 | 0.972 | 0.913 | 0.964 | 0.948 | 1.031 | 0.941 | 0.864 | 1.050 | 1.033 | 0.955 |
| Green | 0.894 | 0.985 | 0.928 | 0.927 | 0.976 | 1.083 | 1.039 | 0.961 | 1.030 | 1.020 | 1.019 |
| Red | 0.857 | 0.996 | 0.947 | 0.941 | 0.982 | 1.103 | 1.065 | 0.996 | 1.019 | 1.012 | 1.027 |
| NIR | 0.823 | 0.975 | 0.947 | 0.976 | 1.006 | 1.096 | 1.054 | 1.001 | 1.043 | 1.028 | 1.005 |
| SWIR1 | 0.764 | 0.958 | 0.955 | 0.904 | 0.967 | 1.070 | 1.051 | 0.995 | 0.988 | 1.021 | |
| SWIR2 | 0.782 | 1.000 | 0.948 | 0.846 | 0.904 | 1.031 | 1.039 | 0.968 | 0.979 | 0.969 | |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Mann, J.; Maddox, E.; Shrestha, M.; Irwin, J.; Czapla-Myers, J.; Gerace, A.; Rehman, E.; Raqueno, N.; Coburn, C.; Byrne, G.; et al. Landsat 8 and 9 Underfly International Surface Reflectance Validation Collaboration. Remote Sens. 2024, 16, 1492. https://doi.org/10.3390/rs16091492
AMA Style
Mann J, Maddox E, Shrestha M, Irwin J, Czapla-Myers J, Gerace A, Rehman E, Raqueno N, Coburn C, Byrne G, et al. Landsat 8 and 9 Underfly International Surface Reflectance Validation Collaboration. Remote Sensing. 2024; 16(9):1492. https://doi.org/10.3390/rs16091492
Chicago/Turabian StyleMann, Joshua, Emily Maddox, Mahesh Shrestha, Jeffrey Irwin, Jeffrey Czapla-Myers, Aaron Gerace, Eon Rehman, Nina Raqueno, Craig Coburn, Guy Byrne, and et al. 2024. "Landsat 8 and 9 Underfly International Surface Reflectance Validation Collaboration" Remote Sensing 16, no. 9: 1492. https://doi.org/10.3390/rs16091492
APA StyleMann, J., Maddox, E., Shrestha, M., Irwin, J., Czapla-Myers, J., Gerace, A., Rehman, E., Raqueno, N., Coburn, C., Byrne, G., Broomhall, M., & Walsh, A. (2024). Landsat 8 and 9 Underfly International Surface Reflectance Validation Collaboration. Remote Sensing, 16(9), 1492. https://doi.org/10.3390/rs16091492
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.