Nothing Special   »   [go: up one dir, main page]
Next Article in Journal
Ozone Detector Based on Ultraviolet Observations on the Martian Surface
Previous Article in Journal
Remote Sensing LiDAR and Hyperspectral Classification with Multi-Scale Graph Encoder–Decoder Network
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 16
Issue 20
10.3390/rs16203913
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Technical Note

Detection of Complex Formations in an Inland Lake from Sentinel-2 Images Using Atmospheric Corrections and a Fully Connected Deep Neural Network

by
Damianos F. Mantsis
1,*,
Anastasia Moumtzidou
1,
Ioannis Lioumbas
2,
Ilias Gialampoukidis
1,
Aikaterini Christodoulou
2,
Alexandros Mentes
2,
Stefanos Vrochidis
1 and
Ioannis Kompatsiaris
1
1
Centre for Research and Technology Hellas, Information Technologies Institute, 57001 Thessaloniki, Greece
2
Thessaloniki Drinking Water & Sewerage Co. S.A. (EYATH S.A.), 54635 Thessaloniki, Greece
*
Author to whom correspondence should be addressed.
Remote Sens. 2024, 16(20), 3913; https://doi.org/10.3390/rs16203913
Submission received: 13 September 2024 / Revised: 16 October 2024 / Accepted: 17 October 2024 / Published: 21 October 2024
(This article belongs to the Section Ocean Remote Sensing)
Browse Figures
Figure 1
<p>Schematic showing the work flow of our methodology.</p> "> Figure 2
<p>North–south cross-section of the 18 January 2021 formation, showing the remote sensing reflectance in all available bands after atmospheric corrections have been applied. For ACOLITE, all corrected bands are displayed, and for iCOR, only the 10 m bands are displayed.</p> "> Figure 3
<p>Visualization of the 18 January 2021 formation with B5 (560 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that the color bar is not the same for all images to improve the direct comparison of formation characteristics after different atmospheric corrections have been applied.</p> "> Figure 4
<p>Visualization of the 18 January 2021 formation with NIR (833 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that for C2RCC, we plot the 865 nm band, given that the NIR band is not available. Lack of data in the polymer illustration indicates that the polymer algorithm overestimates the correction resulting in a negative <math display="inline"><semantics> <msub> <mi>R</mi> <mrow> <mi>r</mi> <mi>s</mi> </mrow> </msub> </semantics></math>, and therefore is masked out.</p> "> Figure 5
<p>Visualization of the 30 December 2017 formation with B4 (665 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that the color bar scale is lower for C2RCC. The feature in the upper right corner represents land.</p> "> Figure 6
<p>(<b>a</b>) False color image where three patches of different sizes (30 December 2017) are “puzzled together” to form one of the patches that is chosen for annotation, (<b>b</b>) formation annotation, (<b>c</b>) DNN prediction after ACOLITE is applied, and (<b>d</b>) DNN prediction without ACOLITE. For (<b>b</b>–<b>d</b>) oil spill pixels appear with yellow.</p> "> Figure 7
<p>(<b>a</b>) False color image of one of the formations (18 January 2021) chosen for annotation, (<b>b</b>) formation annotation, (<b>c</b>) DNN prediction after ACOLITE is applied, and (<b>d</b>) DNN prediction without ACOLITE. For (<b>b</b>–<b>d</b>) oil spill pixels appear with yellow.</p> "> Figure 8
<p>Red–NIR <math display="inline"><semantics> <msub> <mi>R</mi> <mrow> <mi>r</mi> <mi>s</mi> </mrow> </msub> </semantics></math> scatterplot for the formation and clear water pixels that are annotated and used for the DNN training.</p> "> Figure 9
<p>Schematic showing the DNN in three different configurations corresponding to the three types of experiments: (<b>a</b>) Red-NIR, (<b>b</b>) Green-NIR, and (<b>c</b>) 4-Bands.</p> "> Figure 10
<p>(<b>a</b>) Sentinel-2 false color image (green–red–NIR) showing the oil spill case for the 27 February 2017, (<b>b</b>) zoomed area in yellow box, (<b>c</b>) DNN prediction.</p> ">
Versions Notes

Abstract

:
The detection of complex formations, initially suspected to be oil spills, is investigated using atmospherically corrected multispectral satellite images and deep learning techniques. Several formations have been detected in an inland lake in Northern Greece. Four atmospheric corrections (ACOLITE, iCOR, Polymer, and C2RCC) that are specifically designed for water applications are examined and implemented on Sentinel-2 multispectral satellite images to eliminate the influence of the atmosphere. Out of the four algorithms, iCOR and ACOLITE are able to depict the formations sufficiently; however, the latter is chosen for further processing due to fewer uncertainties in the depiction of these formations as anomalies across the multispectral range. Furthermore, a number of formations are annotated at the pixel level for the 10 m bands (red, green, blue, and NIR), and a deep neural network (DNN) is trained and validated. Our results show that the four-band configuration provides the best model for the detection of these complex formations. Despite not being necessarily related to oil spills, studying these formations is crucial for environmental monitoring, pollution detection, and the advancement of remote sensing techniques.

1. Introduction

Major events that result in water body pollution, i.e., oil spills, constitute a major threat for our oceans, coastal habitats and other human activities associated with the ocean, and can result in environmental disasters with severe economical consequences. Some of these are the result of accidents on commercial ships, oil platforms, oil pipe lines, or illegal discharges from tank and engine room cleaning of tankers and commercial ships [1,2,3,4]. The early detection of such events is the primary concern for local authorities, and satellite observations are the primary tool for this due to the large area they can monitor from space. This represents a cost-effective solution as opposed to installing a grid of sensors throughout the reservoir.
Several types of spaceborne remote sensing instruments have been used to detect and monitor oil spills, each one utilizing a specific portion of the electromagnetic spectrum, i.e., infrared (IR), near infrared (NIR), visible, and ultraviolet (UV) [5,6,7]. However, the most common method is Synthetic Aperture Radar (SAR) images [8]. The spaceborne SAR sensor emits polarized waves downward and collects the echo after the waves have interacted with Earth’s surface [1,3]. SAR observations are the most widely used type of data for ocean monitoring and oil spill detection [3,4,6] for several reasons: (a) they can cover large areas of Earth’s surface, (b) they are not affected by sun glitter, (c) they are independent of daylight and can provide images for nighttime as well, and (d) they are independent of weather conditions and can penetrate through thick clouds [1,4,6]. The detection of oil spills can be achieved when the SAR sensor operates in VV or HH polarization mode (co-polarized backscatter), even though the latter is less common [9].
Most of the detection techniques used nowadays for the detection of oil spills usually rely on sophisticated algorithms that can handle the vast volume of remote sensing data that are generated to monitor Earth’s water bodies. This is achieved through algorithms that use supervised learning methods either on SAR or multispectral optical satellite images.
In the case of the SAR images, oil spills appear as dark spots. The problem, however, is that such dark spots can also appear for other natural reasons [10,11,12] and are called oil slick look-alikes. In this case, the AI algorithms address the problem of separating the true oil spills from the look-alikes. This separation is usually performed in three steps. The first step involves the detection of all the dark spots in SAR images. The second step involves the extraction of features from these dark spots, which are then fed to an algorithm (classifier) which classifies the dark spots in one of two groups, oil spill or look-alike [2,13]. The extracted features that represent the signature of each oil spill can be statistical, geometrical, textural or polarimetric [2,14,15]. Furthermore, the classification methods that have been used so far range from Support Vector Machines (SVMs) [9,16], decision tree forests [17,18], and neural networks [19]. Recently, deep learning and, more specifically, convolutional neural networks have been used to detect oil spills with increasing accuracy [20]. For example, [15] have combined a number of polarimetric features to compose false color images that are fed into a deep convolutional neural network with promising results. Furthermore, [21] has used dual-polarimetric Sentinel-1 images, where both amplitude and phase are utilized, along with additional polarimetric information in combination with a DCNN, capturing the fine details of oil spill instances. Other studies use networks that are specifically designed for oil spills, like the Oil Spill Convolutional Network, also with increased accuracy [20]. Other networks, like the oil spill contextual and boundary-supervised detection network (CBD-Net), have been designed to deal with oil spills with extremely irregular shape or fuzzy boundaries, where other traditional SAR techniques struggle [22]. Besides using the more common co-polarization backscatter (VV) from SAR images, other techniques have also emerged, for example, using the co-polarized dual-pol mode (VV+HH) with an artificial neural network, which significantly increases the accuracy of oil spill detection by reducing the false detection of look-alikes as oil spills [23].
Our study focuses on the detection of surface formations in large inland freshwater reservoirs, which were initially hypothesized to be oil spills. Monitoring the water quality of these reservoirs that provide drinking water is a major concern for water utility companies, as these reservoirs are critical for ensuring safe and reliable water supply [24]. In such environments, the spatial scale of these formations is considerably smaller compared to those typically observed in open ocean settings, necessitating different detection methods and analytical approaches. For example, the lack of proper wind conditions in this case does not allow the use of SAR images, and multispectral images are used instead.
Multispectral satellite images are used less frequently for such studies compared to SAR images. For example, Sentinel-2 images have been used to derive the VIS-NIR reflectance ratio index to detect oil spills in an inland lake in Italy [25]. More recent studies have investigated numerous single-band and multichannel ratios to identify the most effective combination for oil–water separability regarding large oil spills over the Persian Gulf [26] and the Indian Ocean [27]. Object-based methods have also been used to detect oil spills using Sentinel-2 images over the Aegean sea [28].
Before implementing deep learning techniques, similar to the ones use for oil spill detection, the reflectance reaching the satellite sensor is corrected to account for the atmospheric effect. This approach represents the novelty of this study, and the purpose is to develop an algorithm that can be implemented operationally and can detect polluting events in freshwater lakes. Figure 1 represents a flowchart which describes the components of the algorithm we implement in this study.
The structure of this paper begins with the introduction. It then describes the location where the study is conducted, and the data implemented. Next, we detail the atmospheric corrections and the deep learning method implemented, followed by the experiments and the results. The paper ends with the conclusions and the discussion.

2. Data and Location

Our study site is an inland artificial lake, the Polyphytos reservoir, located in Northern Greece (40°15’ N, 22°E) at an altitude of 230 m. It is 25 km long and with an average width of 1500–2000 m oriented from SW to NE, and it is surrounded by high mountains. These geographical characteristics along with the complex surrounding topography can have a strong effect on local weather conditions. The Polyphytos Reservoir, which serves as a crucial water source for Thessaloniki, faces numerous anthropogenic pressures, including hydroelectric power generation, agricultural runoff, industrial discharges, and untreated municipal wastewater from surrounding settlements. Thessaloniki Drinking Water & Sewerage Co SA (EYATH SA), responsible for providing drinking water to more than 1M people, has been actively monitoring the water quality of the reservoir using both satellite remote sensing tools [24] and laboratory analyses. Through these efforts, EYATH SA has observed several persistent formations on the water’s surface, including increased chlorophyll-a levels, elevated turbidity, and the presence of hydrocarbons, which cannot yet be attributed to specific sources. These formations may be linked to nutrient influx from agricultural runoff, industrial pollution, physical phenomena like sediment resuspension, or untreated wastewater discharge, but their exact origins remain unclear. The characteristics of these formations display variable sizes (hundreds to thousands of meters) and intensities, with red and green colors indicating the presence of chlorophyll-a, sediments, and hydrocarbons. These formations are primarily seen in the reservoir’s southern region and often coincide with rising water levels. Their persistence suggests links to nutrient inflows or sediment resuspension, but their exact origins remain undetermined, requiring further validation through in situ sampling and satellite monitoring.
Detecting such formations in a small reservoir is different from detecting them in the open sea and has several limitations regarding the data we can use. For example, for the detection of oil spills over large ocean bodies, Sentinel-1 SAR data (10 m resolution) are routinely implemented. However, careful observation of Sentinel-1 images and comparison with true color images (TCIs) did not reveal any areas with reduced co-polarization backscatter, which is characteristic of oil spills or other pollutants. It should be noted that the same images did reveal several dark features over the neighboring Aegean sea. Although these were identified as look-alikes, it shows that the images are suitable for the detection of formations over the Aegean sea but not over our inland lake. This absence of detection capability over the lake is most likely due to the low wind conditions that are unfavorable because they do not allow the formation of capillary waves in the vicinity of these formations, which are necessary for their detection.
This led to the use of the publicly available multispectral Sentinel-2 MSI satellite images (Level1C). Sentinel-2 is a constellation of two sun synchronous polar-orbiting satellites phased at 180° to each other, with a combined 2–5 days revisit cycle, depending on the location. It has 12 bands at horizontal resolutions of 10 m, 20 m, and 60 m. Unlike the traditional detection techniques that use SAR images and basically separate oil spills and pollutants from the look-alikes, in our case, there are no look-alikes. In practice, this simplifies the problem, and we only have to separate the formations from the clear water.
Another limitation for our study was the small number of known formations in our reservoir. This was partly due to a low revisit cycle (5 days), which limited the clear sky images with such formations to only two, i.e., 30 December 2017 with five, and 18 January 2021 with one larger formation. The dataset was developed in collaboration with EYATH.

3. Methodology

3.1. Atmospheric Corrections

The radiance that the satellite sensors measure at the Top-of-Atmosphere (TOA) differs from the surface reflectance due to the influence of the atmosphere. This is attributed to the scattering and absorption of atmospheric molecules and aerosols, and it is called the “atmospheric effect”. This represents noise when observing the Earth’s surface, and must be removed before the image can be processed. For water applications, this correction is even more important due to the relatively low surface reflectance, which also makes the correction more challenging. Thus, algorithms specially designed for water applications must be applied.
Four atmospheric correction algorithms are used in this study (iCOR, ACOLITE, C2RCC, and Polymer). iCOR is an atmospheric correction that uses information regarding the aerosols and the aerosol optical thickness (AOT) to derive the necessary atmospheric components, which are fed to a radiative transfer model to obtain the R r s [29]. One particular element of this algorithm is that the AOT is retrieved by making use of the spectral variability of land pixels within an image, which implies that land pixels must always be present even for water applications. It is also one of the first correction tools designed to account for the adjacency effect, which results in a significant increase in the reflectance quality in the NIR region.
Polymer consists of two models, a polynomial used to model the spectral reflectance of the atmosphere and a water reflectance model that is based on the water chlorophyll concentration and the backscattering coefficient. It is especially effective in decoupling the atmospheric and surface components of the signal from the sun glint, and it uses spectral matching over the whole spectral range (blue to NIR) to do so. Also, the smooth spectral shape of the atmospheric model allows it to fit any spectrally smooth non-water components, such as aerosols or the adjacency effect. Polymer was selected as the best performing algorithm in the ESA project, Ocean Colour Climate Change Initiative (OC-CCI).
On the other hand, Case 2 Regional Coast Color (C2RCC) is a machine learning-based algorithm ideal for predicting the R r s over optically complex waters [30]. A large number of such waters are carefully characterized based on their inherent optical properties (IOPs), as well as the properties of the overlying atmosphere. Then, these are used to parameterize radiative transfer models for both the water and the atmosphere in order to generate a large database of simulated R r s , which is used to train the neural nets.
Finally, ACOLITE, relies on a multiband “dark spectrum fitting” technique (DSF) to choose the optimal aerosol model and band combination for the atmospheric correction [31]. This technique also allows the estimation of sun glint from the SWIR bands, significantly improving the data availability of nadir-viewing sensors like Sentinel-2, Sentinel-3, and Landsat-8, even though improvements in addressing the adjacency effect still need to take place.
The utility of an atmospheric correction algorithm (ACA) for pollutants detection will depend on two things: (a) the technical characteristics of the output, and (b) the quality of the correction (how accurate the correction is). The first encapsulates how many bands are processed and what the horizontal resolution of the output is. The horizontal resolution of the Sentinel-2 bands is not the same for all bands, and it differs between 10, 20, and 60 m grid-spacing, which can be a limiting factor for some applications. For example, C2RCC generates the R r s only for the first eight bands, which includes Narrow NIR (865 nm) but not NIR (832 nm). The generated output is given at 10 m pixels, even for the bands with an original pixel size of 20 or 60 m. In contrast, iCOR generates R r s for all MSI bands, retaining the resolution of the original Sentinel-2 bands. Polymer, on the other hand, generates R r s for all bands at 10 m resolution. The only exception is the 2nd SWIR band (2202 nm), which is not processed at all. Finally, ACOLITE generates R r s for all available bands with a 10 m resolution; the latter, however, is partially true. This is because, even though the generated output is given at 10 m pixel size, the “visual resolution” is the same as the resolution of the initial MSI bands. This means that for a 20 m band like B6 (704 nm), four 10 m pixels forming a square have relatively similar values, visually forming one 20 m pixel, which cannot be distinguished into four 10 m pixels.
When it comes to the detection of our formations, ACOLITE performs well. All formations are clearly represented with anomalous R r s in most bands. This representation, however, is not the same for all bands. For example, the first three bands (443, 493, 560 nm) represent these formations with reduced R r s anomalies compared to the surrounding clear water, and the remaining bands represent the formations with increased anomalies (Figure 2). This is illustrated in the representation of the 18 January 2021 oil spill with the Green (560 nm) and NIR bands (Figure 3 and Figure 4, respectively). This transition, however, from negative to positive anomalies means that there is a wavelength where this transition takes place, and this can result in a lack of detection capability for bands that are close to this spectral threshold. This is exactly what happens for the 30 December 2017 event, where the red band does not exhibit a specific pattern that resembles these formations (Figure 5), indicating a lack of detection capability for that particular event. This does not happen though for the other two cases, where the formation can be clearly identified in the red band. Regarding the ability of the corrected bands to detect the formations, we must look at the band sensitivity. For ACOLITE, the rule is that the highest sensitivity is achieved with NIR and Narrow NIR (Figure 2), indicating their detection potential. In fact, the sensitivity increases as the wavelength increases from 443 nm to the NIR region, and again decreases as we move into the SWIR region.
On the other hand, when it comes to iCOR, for the 18 January 2021 event, the bands show a similar behavior to that of ACOLITE (Figure 2). The formations are represented with reduced values in the lower part of the MSI spectrum, and with increased values in the medium and higher parts, with the maximum again taking part in the NIR region. It is only for the weak formations of 2017 that things differ. In this case, the first three bands (coastal aerosol, blue, and green) exhibit increased values of R r s for the polluted pixels. Such inconsistency can add uncertainty to any detection method. Another limitation of iCOR is the spectral diversity of the processed image that is required for the algorithm to run properly. This is exactly what happened during the 18 January 2021 event, where the land surrounding our lake was fully covered with snow, which may have affected the atmospheric correction in ways that are unknown at this point and therefore cannot be addressed in this study. Regarding the sensitivity of the iCOR corrected bands, it is several times (from 2 to 7) greater compared to that of ACOLITE (Figure 2, Figure 3 and Figure 4) indicating a greater detection potential.
C2RCC on the other hand, identifies the pollutants only with negative anomalies (Figure 2, Figure 3 and Figure 4). However, despite its advantage to output the first eight bands at the highest possible resolution (10 m), it also has severe limitations for this applications. This is because parts of the formations are automatically masked out, especially those where the formations are relatively thicker or more intense. This is particularly the case for the 18 January 2021 event, which gives the formation land-like optical properties, and are identified by C2RCC as land. This is because C2RCC is an atmospheric correction based on neural nets, which are trained based on a large sample of water to estimate the water optical properties suitable to detect water pigments, i.e., chl-a and TSM. Hence, because of this, it is only valid for water bodies. This could be seen as a limitation for our application, given that the most intense parts of the formations are not identified as such. Another disadvantage of C2RCC is that its detection capability varies. For example, it can clearly detect the formation of 18 January 2021 (even by masking it as land), but it is unable to detect the one from 30 December 2017. Overall, it has a maximum sensitivity in the 560 nm range, rapidly decreasing to zero as we move in the red edge and NIR spectral region.
Unlike the first three ACAs, polymer has a much lower ability to detect such formations overall. The pixels identified are represented by rather weak R r s anomalies. Additionally, these values do not exhibit a robust structure (or they do not form a cluster of uniformly anomalous R r s values that contrast the surrounding clear water). This happens across bands, and it can be clearly seen for the 18 January 2021 event (Figure 4), where the pollutants are less defined compared to those seen in R r s i C O R and R r s A C O . However, this is not unjustified, given that polymer relies on a water reflectance model based on Chl and it is simply not designed for polluted water. Both polymer and C2RCC work pixel by pixel, making assumptions about the water reflectance without using the entire MSI image. On the other hand, ACOLITE and iCOR make use of the whole image (land and water).
Given that the ultimate goal is to use R r s to detect formations of unknown origin, the question is which algorithm should be used for the correction, and which bands are best for this. Our results clearly show that only iCOR and ACOLITE corrected bands have the potential to successfully detect the pollution events in question. However, iCOR exhibits some inconsistency regarding the type of anomaly our formations are represented with, which raises the uncertainty level, leading us to suggest ACOLITE as a better atmospheric correction for such applications. Regarding the band choice, this depends on the band resolution; the higher the resolution, the higher its usage, especially for small events where the formation is just a few 10 m pixels across. Therefore, the recommendation is to use the four 10 m resolution bands (red, green, blue, and NIR). However, in the future, the use of the 20 m bands should not be ruled out, given that ACOLITE gives the possibility of using these bands for detection. Bands 783 and 865 μ m are good examples of this, given that they have a strong sensitivity to such formations (Figure 2a) and could be used for larger formations, where the band resolution would not play a compromising role in their use. In the future, we could also combine bands from different algorithms.

3.2. Parameter Learning and Optimization

Our study focuses on formations that were observed during 30 December 2017 and 18 January 2021 in order to thoroughly examine these specific cases. Given the limitation of the small number of formations during these two days, it would be hard to implement an object-based algorithm, which requires a relatively large number of similar formations for training. Therefore, a pixel-based neural network algorithm is implemented instead.
The first step is to create the training dataset. For this, we create false color images (FCI), and we observe the identified formations. The FCI are generated by combining three atmospherically corrected bands (red, green, and NIR), which offer the highest possible resolution and detection capacity. Five orthogonal patches that depict such formations are chosen (roughly 100 × 100 pixels each), three from the first image (30 December 2017) and another two from the second one (18 January 2021). Figure 6a and Figure 7a illustrate two of the formations used in our study, one from each Sentinel-2 image. Then, the pixels corresponding to each formation from each patch are annotated via visual inspection as one, and an equal number of pixels corresponding to clear water are labeled as zero (Figure 6b and Figure 7b). Applying the above results in five groups of pixels representing our formations with 108, 74, 75, 156, and 137 numbers of pixels each. Additionally, given that the first MSI image contains only very small formations, we artificially generate patches that are the result of smaller patches puzzled together to form a larger one for practical reasons (Figure 6a). The annotation applied to these formations cannot include the entire extent of the formations for two reasons. First, some parts exhibit a relatively weak color contrast with the environment and cannot be distinguished from the surrounding clear water. This is true mostly in the periphery of the formations where the pollution layer is relatively thin. Second, the inclusion of pixels with very weak color contrast with clear water could make our model too sensitive and could lead to an increased false positive rate, which has to be avoided.
One thing to note here is that the pixels corresponding to the formations chosen from the two images have different characteristics, forming two distinct groups or R r s values in all bands (Figure 8, scatterplot). This means that the two pixel groups have to be carefully balanced numerically, as otherwise the training of the DNN on R r s values predominantly from one group would result in a model that might fail to detect pixels corresponding to the other group.
We conduct three types of experiments. In the first, two vectors containing the corresponding green and NIR R r s values are fed to the algorithm along with the proper label (Figure 9a). In the second (Figure 9b), we use red and NIR R r s instead and in the third (Figure 9c) we use all four 10 m bands (492, 560, 665, and 832 nm). In each case, we implement a deep neural network (DNN) with two fully connected hidden layers and an output layer. The two hidden layers have 12 and 8 units, respectively, and they use Relu activation. The output layer uses a sigmoid activation and has only two output classes, 0 or 1, which correspond to “no oil spill” or “oil spill”. The DNN is run with eight different learning rates, ranging from 0.00005 to 0.1 (Table 1) and a batch size of 10 and with 100 epochs. These are all hyperparameters of gradient descent that are essential for optimizing the algorithm and more specifically for minimizing the loss function. The learning rate represents the step size at each iteration, the batch size controls the number of training samples to work through before the model’s internal parameters are updated, and the number of epochs controls the number of complete passes through the training dataset. Additionally, two different optimizers are tested, Adam and SGD. Given the fact that every time we run the model, the weights of the neural network are randomly initialized, leading to a different result, we run the model 10 times with the same combination of hyperparameters. At the end, we estimate the final result by averaging our metrics from these 10 different experiments.

4. Results

Initially, we take five identified and annotated formations (observed during 30 December 2017 and 18 January 2021) from the Sentinel-2 images and we operate a fivefold validation process, where each time, we use four of the formations to train the algorithm and the last one is used for testing. We perform this five times, while the formations rotate into testing mode. The metrics we employ to measure the performance of the algorithm are false positive rate, IoU, and accuracy. The results share common characteristics for all three types of experiments. First, only the Adam optimizer performs adequately. On the other hand, the SGD optimizer fails to accurately predict the formations in almost every run, either due to a large number of FPs or due to the complete lack of detection capability. Second, the optimal learning rate is roughly in the range (0.0005–0.001) for the IoU and (0.00005–0.0001) for the FPR.
When it comes to the optimal band combination for detecting the formations, it seems that the four-band combination outperforms any of the two-band combinations, with the red–NIR being the least successful (Table 1). More specifically, the best results (after we average over all formations) that the DNN achieves during training are 0.04, 71.18, and 99.46% for the FPR, IoU, and accuracy, respectively. Figure 6c and Figure 7c show examples of the DNN prediction for two individual formations. A comparison of the annotation with the DNN prediction shows that the model captures the formations very well, also evident in the three metrics used.
Additionally, the perfect model is not expected to have a zero FPR; in fact, a number of FPs are naturally expected, and these represent the prediction of formation pixels that were not included in the annotation. This is perfectly captured in Figure 6a,b, where it is evident that a number of pixels depicting these formations are not included in the annotation and are not included in the training either, leading to a model that fails to detect the formation in its entirety.
Regarding the use of atmospheric correction specifically designed for water applications, the question is whether this step is necessary. To answer this, we perform similar experiments where Level 2 Sentinel-2 images are used instead (called No Correction from now on). Note that for these images, the default algorithm (Sen2cor) is applied. Our results indicate that the use of ACOLITE allows far better metrics (IoU, FPR, and accuracy) during the fivefold validation step (Table 2). Furthermore, the four-band and the green–NIR band combinations perform slightly better compared to the red–NIR configuration, which is in general similar compared to the case where the ACOLITE algorithm is used.
The purpose of this algorithm is to be used operationally to detect similar formations. One of the biggest concerns in such cases is when an algorithm gives false alarms too often, predicting formations when there are none. Therefore, after the validation process and the final training of the algorithm using the optimal hyperparameters and including all formations in our training set, the model is tested for a number of Sentinel-2 images (28 images corresponding to 28 different dates) where no formations are present. The results show that the algorithm does not produce any false positives, indicating its potential for operational use.
To further evaluate the prediction skill of our algorithm, we use the oil spill case from an inland lake in Italy (lake Pertusillo) published by [25]. This event is chosen given its similarity (optically) to our formations. This particular study mentions several Sentinel-2 images that might contain oil spills due to inland oil drilling activity. Before running the DNN algorithm, all images are processed with ACOLITE to correct for the atmospheric effect. It turns out that our trained DNN does have some detection skill, especially for the 27 February 2017 case, where widespread oil contamination is present (and verified) throughout the lake (Figure 10). On the other hand, for other cases mentioned in the [25] study, the model does not exhibit the same detection skill, and it detects no oil spills. It should be noted though that some of these Sentinel-2 images are referred to as “suspect” images and are not accompanied by in situ measurements that could verify the presence of hydrocarbons. Another thing that should be noted is that lake Pertusillo experiences algae blooms during the same time with the confirmed oil spills, affecting the visible part of the spectrum. This could explain why these oil spills do not have the same optical properties as the ones from the Polyphytos reservoir. For example, maximum sensitivity is associated with the red edge bands. Additionally, for the widespread oil spill of 27 February 2017, the boundaries of the oil spill can only be discerned in the red edge and NIR parts of the spectrum. The lack of sensitivity of the visible bands to oil spills could be partly responsible for the lack of detection skill, given that three out of the four bands our algorithm uses are from the VIS spectrum. In conclusion, training a universal algorithm for the detection of such formations in inland waters would require a large number of such events, covering a broad range of water types, which is not available at the moment.

5. Conclusions

This study focuses on the detection of formations of unknown origin (pollutants or even possible oil spills) on an inland lake. We use a pixel-based deep neural network algorithm that uses multispectral images from the Sentinel-2 satellite to detect these formations. Ten meter bands, RGB (visible) and NIR are used in different combinations and are fed into the algorithm as input data. The resulting model is capable of detecting these formations with good accuracy; however, it produces too many false positives by predicting pollutants when there are none. To solve this, we implement atmospheric correction methods that are specifically designed for water applications. Out of the four algorithms tested, ACOLITE is chosen as the most suitable. Its use allows the model to predict the pollutants with greater accuracy as indicated by the metrics used. However, the model appears to be too “stringent“, given that it fails to detect these formations to their full extent. Thus, we conclude that the use of atmospheric correction algorithms in combination with DNNs in general improves the detection capability of the algorithm. Future improvements in the correction method though would certainly benefit the detection by allowing a better annotation and training of the model with more accurate observations.

Author Contributions

Conceptualization, I.G., I.L. and A.C.; methodology, D.F.M.; software, A.M. (Anastasia Moumtzidou); validation, D.F.M.; formal analysis, D.F.M.; investigation, D.F.M.; resources, D.F.M., A.M. (Anastasia Moumtzidou) and I.G.; data curation, A.M. (Anastasia Moumtzidou), I.L., A.C. and A.M. (Alexandros Mentes); writing—original draft preparation, D.F.M.; writing—review and editing, A.M. (Anastasia Moumtzidou), I.G., I.L., A.C. and A.M. (Alexandros Mentes); visualization, D.F.M.; supervision, A.M. (Anastasia Moumtzidou) and I.G.; project administration, S.V. and I.K.; funding acquisition, S.V. and I.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by European Union’s Horizon 2020 Research and Innovation Program aqua3S, under Grant Agreement No 832876.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request.

Conflicts of Interest

Authors Ioannis Lioumpas, Aikaterini Christodoulou, and Alexandros Mentes were employed by the company Thessaloniki Drinking Water & Sewerage Co. S.A. (EYATH S.A.). The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
NIRNear Infrared
SARSynthetic Aperture Radar
DNNDeep Neural Network

References

  1. Brekke, C.; Solberg, A.H. Oil spill detection by satellite remote sensing. Remote Sens. Environ. 2005, 95, 1–13. [Google Scholar] [CrossRef]
  2. Solberg, A.H.; Brekke, C.; Husoy, P.O. Oil spill detection in Radarsat and Envisat SAR images. IEEE Trans. Geosci. Remote Sens. 2007, 45, 746–755. [Google Scholar] [CrossRef]
  3. Topouzelis, K. Oil spill detection by SAR images: Dark formation detection, feature extraction and classification algorithms. Sensors 2008, 8, 6642–6659. [Google Scholar] [CrossRef] [PubMed]
  4. Solberg, A.H. Remote sensing of ocean oil-spill pollution. Proc. IEEE 2012, 100, 2931–2945. [Google Scholar] [CrossRef]
  5. Sun, S.; Hu, C.; Feng, L.; Swayze, G.A.; Holmes, J.; Graettinger, G.; MacDonald, I.; Garcia, O.; Leifer, I. Oil slick morphology derived from AVIRIS measurements of the Deepwater Horizon oil spill: Implications for spatial resolution requirements of remote sensors. Mar. Pollut. Bull. 2016, 103, 276–285. [Google Scholar] [CrossRef]
  6. Fingas, M.F.; Brown, C.E. Review of oil spill remote sensing. Mar. Pollut. Bull. 2014, 83, 9–23. [Google Scholar] [CrossRef]
  7. Fingas, M.F.; Brown, C.E. A review of oil spill remote sensing. Sensors 2017, 18, 91. [Google Scholar] [CrossRef]
  8. Minchew, B.; Jones, C.E.; Holt, B. Polarimetric analysis of backscatter from the Deemwater Horizon oil spills using L-band synthetic aperture radar. IEEE Trans. Geosci. Remote Sens. 2012, 50, 3812–3830. [Google Scholar] [CrossRef]
  9. Singha, S.; Ressel, R.; Velloto, D.; Lehner, S. A combination of traditional and polarimetrric features for oil spill detection using TerraSAR-X. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 4979–4990. [Google Scholar] [CrossRef]
  10. Fingas, M.F.; Brown, C.E. Review of oil spill remote sensing. Spill Sci. Technol. Bull. 1997, 4, 199–208. [Google Scholar] [CrossRef]
  11. Espedal, H.A.; Johannessen, O.M. Cover: Detection of oil spills near offshore installations using synthetic aperture radar (SAR). Int. J. Remote Sens. 1997, 21, 2141–2144. [Google Scholar] [CrossRef]
  12. Kapustin, I.A.; Shomina, O.V.; Ermoshkin, A.V.; Bogatov, N.A.; Kupaev, A.V.; Molkov, A.A.; Ermakov, S.A. On capabilities of tracking marine surface currents using artificial film slicks. Remote Sens. 2019, 11, 840. [Google Scholar] [CrossRef]
  13. Solberg, A.S.; Storvik, G.; Solberg, R.; Volden, E. Automatic detection of oil spills in ERS SAR images. IEEE Trans. Geosci. Remote Sens. 1999, 37, 1916–1924. [Google Scholar] [CrossRef]
  14. Liu, P.; Zhao, C.; Li, X.; He, M.; Pichel, W. Identification of ocean oil spills in SAR imagery based on fuzzy logic algorithm. Int. J. Remote Sens. 2010, 31, 4819–4833. [Google Scholar] [CrossRef]
  15. Guo, H.; Wu, D.; An, J. Discrimination of oil slicks and lookalikes in polarimetric SAR images using CNN. Sensors 2017, 17, 1837. [Google Scholar] [CrossRef]
  16. Konstantinidou, E.E.; Kolokoussis, P.; Topouzelis, K.; Moutzouris-Sidiris, I. An open source approach for oil spill detection using Sentinel-1 SAR images. In Proceedings of the 7th International Conference on Remote Sensing and Geoinformation of the Environment, Paphos, Cyprus, 18–21 March 2019. [Google Scholar]
  17. Singha, S.; Vespe, M.; Trieschmann, O. Automatic Synthetic Aperture Radar based oil spill detection and performance estimation via a semi-automatic operation service benchmark. Mar. Pollut. Bull. 2013, 73, 199–209. [Google Scholar] [CrossRef]
  18. Topouzelis, K.; Psyllos, A. Oil spill feature selection and classification using decision tree forest on SAR image data. J. Photogramm. Remote Sens. 2012, 68, 135–143. [Google Scholar] [CrossRef]
  19. Cantorna, D.; Dafonte, C.; Inglesias, A.; Arcay, B. Oil spill segmentation in SAR images using convolutional neural networks: A comparative analysis with clustering and logistic regression algorithms. Appl. Soft Comput. J. 2019, 84, 105716. [Google Scholar] [CrossRef]
  20. Zeng, K.; Wang, Y. A deep convolutional neural network for oil spill detection from spaceborne SAR images. Remote Sens. 2020, 12, 1015. [Google Scholar] [CrossRef]
  21. Ma, X.; Xu, J.; Wu, P.; Kong, P. Oil spill detection based on deep convolutional neural network using polarimetric scattering information from Sentinel-1 images. IEEE Trans. Geosci. Remote Sens. 2022, 60, 4204713. [Google Scholar] [CrossRef]
  22. Zhu, Q.; Zhang, Y.; Li, Z.; Yan, X.; Guan, Q.; Zhong, Y.; Li, D. Oil spill contextual and boundary-supervised detection network based on marine SAR images. IEEE Trans. Geosci. Remote Sens. 2022, 60, 5213910. [Google Scholar] [CrossRef]
  23. Daeseong, K.; Jung, H.-S. Mapping oil spills from dual-polarized SAR images using an artificial neural network: Application to oil spill in the Kerch Strait in November 2007. Sensors 2018, 18, 2237. [Google Scholar] [CrossRef] [PubMed]
  24. Lioumbas, J.; Christodoulou, A.; Katsiapi, M.; Xanthopoulou, N.; Stournara, P.; Spahos, T.; Seretoudi, G.; Mentes, A.; Theodoridou, N. Satellite remote sensing to improve source water quality monitoring: A water utility’s perspective. Remote Sens. Appl. Soc. Environ. 2023, 10, 101561. [Google Scholar] [CrossRef]
  25. Laneve, G.; Bruno, M.; Mukherjee, A.; Messineo, V.; Giuseppetti, R.; De Pace, R.; D’Ugo, E. Remote sensing detection of algal blooms in a lake impacted by petroleum hydrocarbons. Remote Sens. 2017, 14, 121. [Google Scholar] [CrossRef]
  26. Zakzouk, M.; El-Magd, I.A.; Ali, E.M.; Abdulaziz, A.M.; Rehman, A.; Saba, T. Novel oil spill indices for Sentinel-2 imagery: A case study of natural seepage in Qaruh Island, Kuwait. MethodsX 2024, 12, 102520. [Google Scholar] [CrossRef]
  27. Rajendran, S.; Vethamony, P.; Sadooni, F.N.; Al-Kuwari, H.A.S.; Al-Khayat, J.A.; Govil, H.; Nasir, S. Sentinel-2 image transfromation methods for mapping oil spill-A case study with Wakashio oil spill in the Indian Ocean, off Mauritius. MethodsX 2021, 8, 101327. [Google Scholar] [CrossRef]
  28. Kolokoussis, P.; Karathanassi, V. Oil spill detection and mapping using Sentinel-2 imagery. J. Mar. Sci. Eng. 2018, 6, 4. [Google Scholar] [CrossRef]
  29. De Keukelaere, L.; Sterckx, S.; Adriaensen, S.; Knaeps, E.; Reusen, I.; Giardino, C.; Bresciani, M.; Hunter, P.; Neil, C.; Van der Zande, D.; et al. Atmospheric correction of Landsat-8/OLI and Sentinel-2/MSI data using iCOR algorithm: Validation for coastal and inland waters. Eur. J. Remote Sens. 2018, 51, 525–542. [Google Scholar] [CrossRef]
  30. Brockmann, C.; Doerffer, R.; Peters, M.; Kerstin, S.; Embacher, S.; Ruescas, A. Evolution of the C2RCC neural network for Sentinel-2 and 3 for the retrieval of ocean color products in normal and extreme optically complex waters. In Proceedings of the Living Planet Symposium, Prague, Czech Republic, 9–13 May 2016. [Google Scholar]
  31. Vanhellemeont, Q. Adaptation of the dark spectrum fitting atmospheric correction for aquatic applications of the Landsat and Sentinel-2 archives. Remote Sens. Environ. 2019, 225, 175–192. [Google Scholar] [CrossRef]
Remotesensing 16 03913 g001
Figure 1. Schematic showing the work flow of our methodology.
Figure 1. Schematic showing the work flow of our methodology.
Remotesensing 16 03913 g001
Remotesensing 16 03913 g002
Figure 2. North–south cross-section of the 18 January 2021 formation, showing the remote sensing reflectance in all available bands after atmospheric corrections have been applied. For ACOLITE, all corrected bands are displayed, and for iCOR, only the 10 m bands are displayed.
Figure 2. North–south cross-section of the 18 January 2021 formation, showing the remote sensing reflectance in all available bands after atmospheric corrections have been applied. For ACOLITE, all corrected bands are displayed, and for iCOR, only the 10 m bands are displayed.
Remotesensing 16 03913 g002
Remotesensing 16 03913 g003
Figure 3. Visualization of the 18 January 2021 formation with B5 (560 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that the color bar is not the same for all images to improve the direct comparison of formation characteristics after different atmospheric corrections have been applied.
Figure 3. Visualization of the 18 January 2021 formation with B5 (560 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that the color bar is not the same for all images to improve the direct comparison of formation characteristics after different atmospheric corrections have been applied.
Remotesensing 16 03913 g003
Remotesensing 16 03913 g004
Figure 4. Visualization of the 18 January 2021 formation with NIR (833 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that for C2RCC, we plot the 865 nm band, given that the NIR band is not available. Lack of data in the polymer illustration indicates that the polymer algorithm overestimates the correction resulting in a negative R r s , and therefore is masked out.
Figure 4. Visualization of the 18 January 2021 formation with NIR (833 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that for C2RCC, we plot the 865 nm band, given that the NIR band is not available. Lack of data in the polymer illustration indicates that the polymer algorithm overestimates the correction resulting in a negative R r s , and therefore is masked out.
Remotesensing 16 03913 g004
Remotesensing 16 03913 g005
Figure 5. Visualization of the 30 December 2017 formation with B4 (665 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that the color bar scale is lower for C2RCC. The feature in the upper right corner represents land.
Figure 5. Visualization of the 30 December 2017 formation with B4 (665 nm) after C2RCC, iCOR, Polymer, and ACOLITE corrections have been implemented. Note that the color bar scale is lower for C2RCC. The feature in the upper right corner represents land.
Remotesensing 16 03913 g005
Remotesensing 16 03913 g006
Figure 6. (a) False color image where three patches of different sizes (30 December 2017) are “puzzled together” to form one of the patches that is chosen for annotation, (b) formation annotation, (c) DNN prediction after ACOLITE is applied, and (d) DNN prediction without ACOLITE. For (b–d) oil spill pixels appear with yellow.
Figure 6. (a) False color image where three patches of different sizes (30 December 2017) are “puzzled together” to form one of the patches that is chosen for annotation, (b) formation annotation, (c) DNN prediction after ACOLITE is applied, and (d) DNN prediction without ACOLITE. For (b–d) oil spill pixels appear with yellow.
Remotesensing 16 03913 g006
Remotesensing 16 03913 g007
Figure 7. (a) False color image of one of the formations (18 January 2021) chosen for annotation, (b) formation annotation, (c) DNN prediction after ACOLITE is applied, and (d) DNN prediction without ACOLITE. For (b–d) oil spill pixels appear with yellow.
Figure 7. (a) False color image of one of the formations (18 January 2021) chosen for annotation, (b) formation annotation, (c) DNN prediction after ACOLITE is applied, and (d) DNN prediction without ACOLITE. For (b–d) oil spill pixels appear with yellow.
Remotesensing 16 03913 g007
Remotesensing 16 03913 g008
Figure 8. Red–NIR R r s scatterplot for the formation and clear water pixels that are annotated and used for the DNN training.
Figure 8. Red–NIR R r s scatterplot for the formation and clear water pixels that are annotated and used for the DNN training.
Remotesensing 16 03913 g008
Remotesensing 16 03913 g009
Figure 9. Schematic showing the DNN in three different configurations corresponding to the three types of experiments: (a) Red-NIR, (b) Green-NIR, and (c) 4-Bands.
Figure 9. Schematic showing the DNN in three different configurations corresponding to the three types of experiments: (a) Red-NIR, (b) Green-NIR, and (c) 4-Bands.
Remotesensing 16 03913 g009
Remotesensing 16 03913 g010
Figure 10. (a) Sentinel-2 false color image (green–red–NIR) showing the oil spill case for the 27 February 2017, (b) zoomed area in yellow box, (c) DNN prediction.
Figure 10. (a) Sentinel-2 false color image (green–red–NIR) showing the oil spill case for the 27 February 2017, (b) zoomed area in yellow box, (c) DNN prediction.
Remotesensing 16 03913 g010
Table 1. IoU and FPR results corresponding to the validation runs for the 4 bands, red–NIR and green–NIR experiments. Values represent averages across the five rotational runs.
Table 1. IoU and FPR results corresponding to the validation runs for the 4 bands, red–NIR and green–NIR experiments. Values represent averages across the five rotational runs.
4 BandsGreen–NIRRed–NIR
LRIoUFPRIoUFPRIoUFPR
0.17.4830.57.4128.031.4832
0.0512.730.4315.8330.131818.26
0.0143.081.7954.640.7454.370.82
0.00546.841.862.940.6358.150.93
0.00160.430.9369.820.4263.090.8
0.000571.180.468.760.2758.410.65
0.000133.780.0540.880.0531.090.21
0.0000526.440.0329.920.0528.310.04
Table 2. Same as Table 1 but for experiments where Sen2cor atmospheric correction is used.
Table 2. Same as Table 1 but for experiments where Sen2cor atmospheric correction is used.
4 BandsGreen–NIRRed–NIR
LRIoUFPRIoUFPRIoUFPR
0.125.1110.7430.435.2125.248.84
0.0532.111.0737.95.4830.817.16
0.0140.482.2238.52.2329.434.06
0.00538.522.5938.922.0930.664
0.00141.192.2139.042.1331.983.34
0.000540.672.1238.432.0732.483.33
0.000146.731.6744.621.7444.012.37
0.0000546.721.5546.781.6442.612.91
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Mantsis, D.F.; Moumtzidou, A.; Lioumbas, I.; Gialampoukidis, I.; Christodoulou, A.; Mentes, A.; Vrochidis, S.; Kompatsiaris, I. Detection of Complex Formations in an Inland Lake from Sentinel-2 Images Using Atmospheric Corrections and a Fully Connected Deep Neural Network. Remote Sens. 2024, 16, 3913. https://doi.org/10.3390/rs16203913

AMA Style

Mantsis DF, Moumtzidou A, Lioumbas I, Gialampoukidis I, Christodoulou A, Mentes A, Vrochidis S, Kompatsiaris I. Detection of Complex Formations in an Inland Lake from Sentinel-2 Images Using Atmospheric Corrections and a Fully Connected Deep Neural Network. Remote Sensing. 2024; 16(20):3913. https://doi.org/10.3390/rs16203913

Chicago/Turabian Style

Mantsis, Damianos F., Anastasia Moumtzidou, Ioannis Lioumbas, Ilias Gialampoukidis, Aikaterini Christodoulou, Alexandros Mentes, Stefanos Vrochidis, and Ioannis Kompatsiaris. 2024. "Detection of Complex Formations in an Inland Lake from Sentinel-2 Images Using Atmospheric Corrections and a Fully Connected Deep Neural Network" Remote Sensing 16, no. 20: 3913. https://doi.org/10.3390/rs16203913

APA Style

Mantsis, D. F., Moumtzidou, A., Lioumbas, I., Gialampoukidis, I., Christodoulou, A., Mentes, A., Vrochidis, S., & Kompatsiaris, I. (2024). Detection of Complex Formations in an Inland Lake from Sentinel-2 Images Using Atmospheric Corrections and a Fully Connected Deep Neural Network. Remote Sensing, 16(20), 3913. https://doi.org/10.3390/rs16203913

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop