A new co-learning method in spatial complex fuzzy inference systems for change detection from satellite images

The detection of spatial and temporal changes (or change detection) in remote sensing images is essential in any decision support system about natural phenomena such as extreme weather conditions, climate change, and floods. In this paper, a new method is proposed to determine the inference process parameters of boundary point, rule coefficient, defuzzification coefficient, and dependency coefficient and present a new FWADAM+ method to train that set of parameters simultaneously. The initial data are clustered simultaneously according to each data group. This result will be the basis for determining a suitable set of parameters by using the FWADAM+ concurrent training algorithm. Eventually, these results will be inherited in the following data groups to build other complex fuzzy rule systems in a shorter time while still ensuring the model’s efficiency. The weather imagery database of the United States Navy (US Navy) is used to evaluate and compare with some related methods using the root-mean-squared error (RMSE), R-squared (R²) measures, and the analysis of variance (ANOVA) model. The experimental results show that the proposed method is up to 30% better than the SeriesNet method, and the processing time is 10% less than that of the SeriesNet method.

The data that support the findings of this study are available from US Navy but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of US Navy.

1. We will represent the loss function (20) in the Appendix A.

This research has been funded by the Research Project: VAST01.07/22-23, Vietnam Academy of Science and Technology.

Authors and Affiliations

1. Quality Assurance Center, Hanoi University of Industry, Hanoi, Vietnam

   Le Truong Giang & Nguyen Van Luong

2. Graduate University of Science and Technology, Vietnam Academy of Science and Technology, Hanoi, Vietnam

   Le Truong Giang

3. VNU Information Technology Institute, Vietnam National University, Hanoi, Vietnam

   Le Hoang Son

4. Institute of Information Technology, Vietnam Academy of Science and Technology, Hanoi, Vietnam

   Nguyen Long Giang

5. Faculty of Computer Science and Engineering, Thuyloi University, Hanoi, Vietnam

   Tran Manh Tuan

6. Institute of Techniques for Special Engineering, Le Quy Don Technical University, Hanoi, Vietnam

   Mai Dinh Sinh

7. Faculty of Business and Management, UCSI University, Jalan Menara Gading, 56000, Cheras, Kuala Lumpur, Malaysia

   Ganeshsree Selvachandran

8. Department of Digital Systems, University of Thessaly, 41500, Larissa, Greece

   Vassilis C. Gerogiannis

Corresponding author

Correspondence to Le Hoang Son.

Conflict of interest

The authors declare that they do not have any conflict of interests. All authors have checked and agreed the submission.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A: Details of loss function

Appendix B: Numerical example

Step 1: Input data preprocessing.

Step 1.1. Convert satellite image from color image to gray image.

Let us say we have 10 RGB images, each with a size 9 × 9. (Because the formula applies to all ten images is the same. Therefore, we will show the example with two images to avoid duplication in the information.)

To perform the conversion to grayscale, we will follow the formula:

where Y is the corresponding gray pixel value, R, G, and B are, respectively, the pixel value of the channels: red, green, and blue of the color image.

Then, suppose our grayscale data is obtained as follows:

X₁:

X₂:

Step 1.2: Reduce image size by representative pixels.

From the input image, we will proceed to group pixels. (In this case, we will choose the value c = 3, which means we will group the 3 × 3 small images corresponding to the color we have presented above.)

From the above data, according to formula number (3), we can determine Im^tb and \(\kappa\) as follows:

\({\text{Im}}_{{^{1} }}^{tb}\):

\({\text{Im}}_{{^{2} }}^{tb}\):

\(\kappa_{1}\):

\(\kappa_{2}\):

From here, we can determine \({\text{I}} \overline{m}\) according to formula number (2)

Similarly, we get the following input:

Step 1.3: Determine the difference matrix (imaginary part).

Imaginary part (is the difference matrix): Determined by directly subtracting the difference between the corresponding regions of the remote sensing image according to formula number (3):

Step 1.4: Data sampling.

N = 10 Input images from Images 1 to 10 as above, we choose sample size \(\kappa = 4\), apply formula number (4), we get: \(M = \frac{10 - 4}{{4\left( {1 - 0.5} \right)}} + 1 = 4\).

Sample 1:

Training: From Image 1 to 3; Validation: Image 3; Testing: Image 4.

Sample 2:

Training: From Image 3 to 5; Validation: Image 5; Testing: Image 6.

Sample 3:

Training: From Image 5 to 7; Validation: Image 7; Testing: Image 8.

Sample 4.

Training: From Image 7 to 9: Validation: Image 9; Testting: Image 10.

Step 2: Using fuzzy C-means to cluster the input data with both the real and the imaginary parts of each sample.

With sample 1:

Using FCM clustering algorithm to cluster simultaneously with \({\text{I}} \overline{m}^{(t)}\) and HOD get the corresponding pairs of values.

• Number of cluster: 2

• Value m = 2

• EPS (Threshold of difference between two consecutive iterations) = 0.001

• Number of iterations t: 3

Output:

• Membership matrix: U

• Center vector: V

Step 2.1: Transform X ^t) value and HOD in range [0, 1]

Step 2.2: Initialize the matrix of values Vector cluster center according to random values

Satisfied:

• \(V_{j} :\,\) is the center vector

  $$ V_{j1} \in \left( {\min Xi,\ldots\max Xi} \right);V_{j2} \in \left( {\min HODi,\ldots\max HODi} \right) $$
  $$ V_{1}^{(0)} = \,\left[ {\begin{array}{*{20}r} \hfill {0.1416} \\ \hfill {0.0024} \\ \end{array} } \right];\quad V_{2}^{(0)} = \,\left[ {\begin{array}{*{20}r} \hfill {0.1744} \\ \hfill {0.0113} \\ \end{array} } \right] $$

Step 2.3: Calculate U by center vector V

Step 2.4: Update Center vector V

Calculate the different between \(V^{(1)}\) and \(V^{(0)}\) using Euclid distance

Step 2.5: Repeat step 2.3 and step 2.4 while still satisfying one of two conditions.

• Condition 1: The number of iterations is less than the maximum number of iterations (3)

• Condition 2: \(\left\| {V^{(t)} \, - \,V^{(t - 1)} \,} \right\|\,\, \le \,\,EPS\left( {0.001} \right)\)

Current iteration count = 1 and \(\left\| {V^{(1)} \, - \,V^{(0)} \,} \right\|\, = 0.5665 > EPS\).

Continue to the second iteration:

Calculate the different between \(V^{(2)}\) and \(V^{(1)}\) by using Euclid distance

Current iterations count = 2 and \(\left\| {V^{(2)} \, - \,V^{(1)} \,} \right\|\, = 0.5097\, > \,EPS\).

Continue to the third iterations:

Calculate different between \(V^{(3)}\) and \(V^{(2)}\) by using Euclid distance

Current iteration count = 3 and \(\left\| {V^{(3)} - V^{(2)} } \right\| = 0.0{472} > EPS\).

\(\to\) Stopped due to the number of iterations exceeded.

The result:

Same with the rest of the samples (Sample 2, Sample 3).

Step 3: Generate and Aggregate Spatial CFIS+ rules from clustering results.

1. (a)

   Rule generation with sample 1 and Image X ₁ :

Initialize with all values \(\alpha_{j} = 1;j \in 1,2,\ldots R;\alpha_{j} \in \left( {\alpha_{j}^{a} ,\alpha_{j,}^{b} \alpha_{j}^{c} ,\alpha_{j}^{{a^{\prime}}} ,\alpha_{j}^{{b^{\prime}}} ,\alpha_{j}^{{c^{\prime}}} } \right)\).

Applying formulas (5–10) we get:

Similar to the rest of imaginary part of the remaining data, we have the following rules for the first input data.

Rule 1

Include six parameters a, b, c and \(a^{\prime } , \, b^{\prime } , \, c^{\prime }\).

a, b, c are the coordinates of the first triangle of the real part and \(a^{\prime } , \, b^{\prime } , \, c^{\prime }\) are the coordinates of the first triangle of the imaginary part (see Fig. 

Rule space 1 spatial CFIS of image X₁

Full size image

12).

The value area is in the bottom surface \(({\mathbf{AA}}^{\prime } {\mathbf{C}}^{\prime } {\mathbf{BC}})\) where:

Rule 2

Include six parameters a, b, c and \(a^{\prime } , \, b^{\prime } , \, c^{\prime }\).

a, b, c are the coordinates of the second triangle of the real part and \(a^{\prime } , \, b^{\prime } , \, c^{\prime }\) are the coordinates of the second triangle of the imaginary part (see Fig. 

Rule space 2 spatial CFIS of image X₁

Full size image

13).

The value area is in the bottom surface \(({\mathbf{AA}}^{\prime } {\mathbf{C}}^{\prime } {\mathbf{BC}})\) where

Same with the rest of image:

• Sample 1 Image X_1, X₂, X₃

• Sample 2 Image X₃, X₄, X₅

• Sample 3 Image X₅, X₆, X₇

• Sample 4 Image X₇, X₈, X₉

• Rule aggregate

At each image of a sample set, rules will be generated and those rules will be added to the rule set of each sample. Each template will have its own set of rules, but the parameters will be inherited from the previous template.

Example: With sample number 1 when initializing values for all parameters, but to pattern number 2 will not initialize the value but use the parameter value obtained from pattern 1.

Step 4: Calculate the interpolated value and synthesize the next predicted Image.

Step 4.1. Inference membership function.

Based on the complex fuzzy rule system in triangular space (Spatial CFIS), determine the value of the membership function of Image \(X_{1}^{\prime }\)

• - With the first pixel (0.607, 0.0111) and rule 1.

•  + We call the first pixel in the rule space D, which will have the following value D (0.607, 0.0111). (Point D on the bottom surface AA′C′BC).

•  + Since point D is outside the bounds, we need to move point D into the 1st rule space with the coefficient \(\mu = \,1.7\)

•  + Draw line BD intersecting line AA′ at point E.

•  + Let F be a point satisfying the condition F in the plane AA′B′ and DF perpendicular to the base.

Then, the height DF is the value of the degree of belonging U of the first pixel D (0.607, 0.0111) into the first rule space (see Fig. 14).

Interpolation of an image point in the first rule space \(\frac{DF}{{BB^{\prime } }} = \frac{DE}{{BE}}\, \to \,DF = \frac{{BB^{\prime} \times DE}}{BE} = \frac{1 \times 0.0087}{{0.1093}} = 0.0796\)

Full size image

• - With the first image point (0.6181, 0.0141) and the second rule.

•  + We call the second Image point in the rule space D, which will have the following value D (0.607, 0.0111). (Point D is on the bottom surface of AA′C′BC.)

•  + Since point D is outside the bounds, we need to move point D into 2nd rule space with \(\mu = \,1.7\)

•  + Draw line BD intersecting line AA′ at point E.

•  + Let F be the point satisfying the condition F on the plane AA′B′ and DF perpendicular to the base surface.

Then, the height DF is the value of the degree of belonging U of the first pixel D (0.607, 0.0111) into the second rule space (see Fig. 15).

Interpolation of an image point in the second rule space

Full size image

Same with the rest of the points.

• - With the second input value pair (0.443, 0.1246).

•  + The degree of belonging of the second pixel to the first rule, DF = 0.3693.

•  + The degree of belonging of the second pixel to the second rule, DF = 0.5375.

• - With the third input value pair (0.5162, 0.0579).

•  + The degree of belonging of the third pixel to the first rule, DF = 0.179.

•  + The degree of belonging of the third pixel to the second rule, DF = 0.2116.

• - With the fourth input value pair (0.2946, 0.2018).

•  + The degree of belonging of the fourth pixel to the first rule, DF = 0.2457.

•  + The degree of belonging of the fourth pixel to the second rule, DF = 0.3952.

• - With the fifth input value pair (0.322, 0.1682).

•  + The degree of belonging of the fifth pixel to the first rule, DF = 0.2684.

•  + The degree of belonging of the fifth pixel to the second rule, DF = 0.4319.

• - With the sixth input value pair (0.4962, 0.0136).

•  + The degree of belonging of the sixth pixel to the first rule, DF = 0.1831.

•  + The degree of belonging of the sixth pixel to the second rule, DF = 0.127.

• - With the seventh input value pair (0.5121, 0.0765).

•  + The degree of belonging of the seventh pixel to the first rule, DF = 0.2758.

•  + The degree of belonging of the seventh pixel to the second rule, DF = 0.3064.

• - With the eighth input value pair (0.5414, 0.0526).

•  + The degree of belonging of the eighth pixel to the first rule, DF = 0.1564.

•  + The degree of belonging of the eighth pixel to the second rule, DF = 0.1951.

• - With the ninth input value pair (0.3883, 0.1166).

•  + The degree of belonging of the ninth pixel to the first rule, DF = 0.3238.

•  + The degree of belonging of the ninth pixel to the second rule, DF = 0.4673.

Step 4.2. Determine rule coefficient \(\beta_{i}\).

Initialized the value \(\beta_{ij} = 1,\,\forall i \in 1,2,\ldots L;j \in 1,2,\ldots R + 1\) and applied formula (12), the attribute values of \(W_{ij}\) correspond to the interpolation values of each pixel into that rule, we have:

The first pixel:

The second pixel:

The third pixel:

The fourth pixel:

The fifth pixel:

The sixth pixel:

The seventh pixel:

The eighth pixel:

The ninth pixel:

Step 4.3. Determine defuzzification coefficient.

Initialized \(\left( {h_{1j} , \, h_{2j} , \, h_{3j} , \, h^{\prime}_{1j} , \, h^{\prime}_{2j} , \, h^{\prime}_{3j} } \right);\,\forall j \in 1,2,\ldots ,\,R\) according to \(\left( {1,2,1,1,2,1} \right)\) and applied formula (13, 14), we have:

DEF of rule 1:

DEF of rule 2:

Step 4.4. Determine dependence coefficient \(\gamma\).

(*) The next Image prediction result of the real part is determined by formula ( 16 ) , as follows:

Similarly, we have:

The inferred value of the predicted image \(O_{{i.{\text{Img}}}}^{*}\) is based on the difference value calculated by formula ( 18 ) , as follows:

Similarly, we have:

(**) The phase part prediction result \(O_{{i.{\text{Img}}}}^{{*^{\prime}}}\) is calculated based on formula ( 17 ) , as follows:

Similarly, we have:

Initialize initial value \(\gamma\) = 0.5. The forecast image of the final representative image is the result of the next image prediction \(O_{i}^{*}\) calculated based on the combined result of the real part and phase prediction pixel according to formula (15), as follows:

Similarly, we have:

Return to normal space

Step 4.5. Final predict result.

After having the final prediction result of the representative image point \(O_{i}^{*}\), proceed to calculate the neighborhood points of each central representative Image point according to formula (19), we get:

With the point image represents \(O_{1}^{*}\)

From \(O_{1}^{*}\) , restore nine neighboring image points as follows:

Proceed with the remaining representative image points \(O_{2}^{*} ,O_{3}^{*} ,O_{4}^{*} ,O_{5}^{*} ,O_{6}^{*} ,O_{7}^{*} ,O_{8}^{*} \,{\text{and}}\,O_{9}^{*}\). We get the next prediction by image as follows:

Step 5: Simultaneous training of the parameters in the model (Co-learning)

After training process, we had a suitable set of parameters X for the next iteration.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions