Nothing Special   »   [go: up one dir, main page]

Next Article in Journal
Multiscale Entropy Analysis: Application to Cardio-Respiratory Coupling
Previous Article in Journal
A Reliable Auto-Robust Analysis of Blood Smear Images for Classification of Microcytic Hypochromic Anemia Using Gray Level Matrices and Gabor Feature Bank
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Machine Learning for Modeling the Singular Multi-Pantograph Equations

by
Amirhosein Mosavi
1,2,
Manouchehr Shokri
3,
Zulkefli Mansor
4,
Sultan Noman Qasem
5,6,
Shahab S. Band
7,8,* and
Ardashir Mohammadzadeh
9
1
Environmental Quality, Atmospheric Science and Climate Change Research Group, Ton Duc Thang University, Ho Chi Minh City, Vietnam
2
Faculty of Environment and Labour Safety, Ton Duc Thang University, Ho Chi Minh City, Vietnam
3
Faculty of Civil Engineering, Institute of Structural Mechanics (ISM), Bauhaus-Universität Weimar, 99423 Weimar, Germany
4
Fakulti Teknologi dan Sains Maklumat, Universiti Kebangsan Malaysia, Bangi 43600, Selangor, Malaysia
5
Computer Science Department, College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia
6
Computer Science Department, Faculty of Applied Science, Taiz University, Taiz 6803, Yemen
7
Future Technology Research Center, National Yunlin University of Science and Technology, 123 University Road, Section 3, Douliou, Yunlin 64002, Taiwan
8
Institute of Research and Development, Duy Tan University, Da Nang 550000, Vietnam
9
Electrical Engineering Department, University of Bonab, Bonab 5551785176, Iran
*
Author to whom correspondence should be addressed.
Entropy 2020, 22(9), 1041; https://doi.org/10.3390/e22091041
Submission received: 27 July 2020 / Revised: 10 August 2020 / Accepted: 11 August 2020 / Published: 18 September 2020
Figure 1
<p>Block diagram of the proposed solver.</p> ">
Figure 2
<p>The structure of suggested T2-FLS.</p> ">
Figure 3
<p>Example 1: (<b>a</b>): Solution performance; (<b>b</b>): Weights of NN.</p> ">
Figure 4
<p>Example 1: (<b>a</b>): Absolute error; (<b>b</b>): The values of FIT, RMSE, VAR and TIC.</p> ">
Figure 5
<p>Example 1: (<b>a</b>): The value of TIC at each iteration; (<b>b</b>): Histogram plot for TIC; (<b>c</b>): Box plot for TIC.</p> ">
Figure 6
<p>Example 1: (<b>a</b>): The value of RMSE at each iteration; (<b>b</b>): Histogram plot for RMSE; (<b>c</b>): Box plot for RMSE.</p> ">
Figure 7
<p>Example 1: (<b>a</b>): The value of VAR at each iteration; (<b>b</b>): Histogram plot for VAR; (<b>c</b>): Box plot for VAR.</p> ">
Figure 8
<p>Example 1: (<b>a</b>): The value of FIT at each iteration; (<b>b</b>): Histogram plot for FIT; (<b>c</b>): Box plot for FIT.</p> ">
Figure 9
<p>Example 2: (<b>a</b>): Solution performance; (<b>b</b>): Weights of NN.</p> ">
Figure 10
<p>Example 2: (<b>a</b>): Absolute error; (<b>b</b>): The values of FIT, RMSE, VAR and TIC.</p> ">
Figure 11
<p>Example 2: (<b>a</b>): The value of TIC at each iteration; (<b>b</b>): Histogram plot for TIC; (<b>c</b>): Box plot for TIC.</p> ">
Figure 12
<p>Example 2: (<b>a</b>): The value of RMSE at each iteration; (<b>b</b>): Histogram plot for RMSE; (<b>c</b>): Box plot for RMSE.</p> ">
Figure 13
<p>Example 2: (<b>a</b>): The value of VAR at each iteration; (<b>b</b>): Histogram plot for VAR; (<b>c</b>): Box plot for VAR.</p> ">
Figure 14
<p>Example 2: (<b>a</b>): The value of FIT at each iteration; (<b>b</b>): Histogram plot for FIT; (<b>c</b>): Box plot for FIT.</p> ">
Versions Notes

Abstract

:
In this study, a new approach to basis of intelligent systems and machine learning algorithms is introduced for solving singular multi-pantograph differential equations (SMDEs). For the first time, a type-2 fuzzy logic based approach is formulated to find an approximated solution. The rules of the suggested type-2 fuzzy logic system (T2-FLS) are optimized by the square root cubature Kalman filter (SCKF) such that the proposed fineness function to be minimized. Furthermore, the stability and boundedness of the estimation error is proved by novel approach on basis of Lyapunov theorem. The accuracy and robustness of the suggested algorithm is verified by several statistical examinations. It is shown that the suggested method results in an accurate solution with rapid convergence and a lower computational cost.

1. Introduction

The application of multi-pantograph differential equations (MDEs) is expanding into various branches of science such as modeling of cell-growth [1], electrodynamics [2], number theory [3], electrodynamics, astrophysics [4], atomic physics [5], among many others.
Recently, due to the importance of MDEs, the solving of these equations have been frequently considered in the literature and many numerical and analytical methods have been presented [6]. For example, in References [7,8,9], a homotopy approach and power series are developed for solving linear MDEs and coinciding of the estimated solution with the exact solution is investigated. In Reference [10], the spectral tau method is studied and the convergence of the presented approach is investigated by L 2 norm. In Reference [11], by obtaining the fractional integral of Taylor wavelets in the sense of Riemann-Liouville definition, an estimated solution is presented for fractional MDEs. In Reference [12], by the Bessel and block-pulse functions, a numerical solution is suggested and it is shown that the increasing of Bessel functions improves the accuracy. In Reference [13], on the basis of topological degree theorem, an analytical method is developed. In Reference [14], by the use of residual power series, an analytical solution is obtained and the efficiency of using residual power series is compared with Chebyshev and Boubaker polynomials. In Reference [15], Adomian decomposition approach is used to construct a solution algorithm for MDEs with fractional-order in the sense of Caputo definition. In Reference [16], by the use of Riemann Liouville fractional derivative and integral definitions, some operational matrices are constructed and then on basis of Jacobi polynomials, an analytical solution is presented. In Reference [17], the collocation method by the use of Boubaker polynomials is developed to convert the problem into a nonlinear system, and then by solving the reduced nonlinear system, an approximated solution is presented. In Reference [18], the spectral tau approach using Jacobi polynomials is improved to solve MDEs. The shifted Gegenbauer-Gauss collocatio technique is introduced in Reference [19], for functional-differential equations. The exponential Jacobi spectral and Jacobi collocation approaches are developed in References [20,21].
Recently, fuzzy logic systems (FLSs) and machine learning algorithms are widely applied on engineering problems [22,23,24,25,26,27,28]. However, to the best knowledge of the authors, the solution of singular MDEs by FLSs and machine learning algorithms has not been studied in the literature. However, quite rarely, some neural methods have been presented for conventional MDEs. For example, in Reference [29], a neural network (NN) is learned by genetic algorithm to find a solution for a pantograph system. In Reference [30], a simple NN is used to find an approximated solution for ordinary differential equations and its accuracy is compared with the analytical solution. In Reference [31], by the method of Lagaris et al, a neural approach is developed for solving MDEs and its efficiency is proved. Now days, the computation techniques in software engineering are used in various branch of scientific problem such as multimedia systems [32,33], security systems [34], forecasting problems [35], stock market prediction [36], control systems [37], internet of things [38], and so on. However these effective techniques quite rarely are applied on MDEs.
Considering the above motivation, in this paper a new approach using T2-FLSs is presented for the solving of singular MDEs. Unlike the aforementioned NN-based methods [29,30,31], the optimization is done by a low computation cost and stable algorithm. The proposed learning algorithm is on the basis of stable SCKF. For the first time, a new approach on the basis of the Lyapunov theorem is suggested to analyze the convergence and closed-loop stability. By several statistical examinations, such as root mean square error (RMSE), inequality coefficient of Theil index (TIC), variance (VAR), fitness (FIT), interquartile range (IR), median (Med), minimum (Min) and mean of absolute error, the accuracy of the suggested method is shown. The main contributions are:
  • A new numerical method is proposed for solving singular MDEs.
  • For the first time, a type-2 fuzzy logic based approach is formulated to find an approximated solution.
  • A new approach on the basis of the Lyapunov theorem is introduced for convergence and stability analysis.
  • Square root cubature Kalman filter is developed for the optimization of the suggested solver.
  • Several statistical examinations are presented to demonstrate the accuracy and stability.
The paper organization is as follows. The problem is formulated in Section 2. The suggested T2-FLS is illustrated in Section 3. The learning algorithm is presented in Section 4. The stability is investigated in Section 5. The evaluation indexes are described in Section 6. The simulation results are provided in Section 7, and finally the main outcomes are summarized in Section 8.

2. Problem Formulation

The suggested solver is designed on the basis of fuzzy systems and SCKF. The general diagram of the suggested solution approach is shown in Figure 1. The problem is described as:
χ ¨ t + k = 1 n χ ˙ r k t / P k t + χ t / G k = F t ,
where the initial conditions are χ 0 = a 1 and χ ˙ 0 = a 2 . F k t and G t are nonlinear functions. If there is a singularity in F k t and G t , then both sides of (1) are multiplied by F k t and G t . The parameters of T2-FLS should be learned such that the estimated solution χ ^ t is to be converged to the exact solution χ t . Then the estimated χ ^ t satisfies:
χ ^ ¨ t + k = 1 n χ ^ ˙ γ k t / P k t + χ ^ t / G k = F t .
The cost function is defined as follows:
J = 1 N i = 1 N χ ^ ¨ i t + k = 1 n χ ^ ˙ i γ k t / P k , i t + χ ^ i t / G i F t 2 + 1 2 χ ^ 0 2 + χ ^ ˙ 0 2 ,
where i = 1 , , N and N is the number of samples.

3. T2-FLS Structure

The structure of T2-FLS χ ^ t is shown in Figure 2. The details are given as follows:
(1)
Get the input t.
(2)
The input t is mapped into time range 0 , 1 .
(3)
The 0 , 1 is divided into M section and for each section a Gaussian membership function (MF) with mean m l , l = 1 , , M and variance ν l is considered.
(4)
The upper and lower firing rules are computed as:
f ¯ l t = exp t m l 2 ν ¯ l 2 , l = 1 , , M ,
f ̲ l t = exp t m l 2 ν ̲ l 2 , l = 1 , , M .
(5)
The normalized rule firings (type-reduction by the Nie-Tan approach [39]) are obtained as:
ψ l t = f ¯ l t + f ̲ l t l = 1 M f ¯ l t + f ̲ l t .
(6)
The output is obtained as:
χ ^ t , θ , m , ν = l = 1 M θ l ψ l t , m , ν ,
where M is number of rules, θ is the vector of rule parameters. From (7), χ ^ ˙ is computed as:
χ ^ ˙ t , θ , m , ν = l = 1 M θ l f ˙ l t l = 1 M f l t f l t l = 1 M f ˙ l t l = 1 M f l t 2 .
From (8), f ˙ l t is:
f ˙ l t = 2 t m l ν l 2 exp t m l 2 ν l 2 .
Equation (9), can be rewritten as:
f ˙ l t = 2 t m l ν l 2 f l t .
Then from (8) and (10), χ ^ ˙ t , θ , m , ν is rewritten as:
χ ^ ˙ t , θ , m , ν = l = 1 M θ l 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M f l t .
Similarly, from (11), χ ^ ¨ t , θ , m , ν is obtained as:
χ ^ ¨ t , θ , m , ν = l = 1 M θ l 2 t ν l 2 + f ˙ l t l = 1 M 2 t m l ν l 2 + f l t l = 1 M 2 t ν l 2 l = 1 M f l t 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M f ˙ l t / l = 1 M f l t 2 .
Considering (10), the Equation (12) is rewritten as:
χ ^ ¨ t , θ , m , ν = l = 1 M θ l 2 t ν l 2 + f ˙ l t l = 1 M 2 t m l ν l 2 + f l t l = 1 M 2 t ν l 2 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M 2 t m l ν l 2 / l = 1 M f l t .

4. Learning Method

The suggested T2-FLS is optimized through the SCKF. To apply SCKF on learning of T2-FLS such that the cost function (3) to be minimized, the following state-space representation is taken to account:
ξ k + 1 = ξ k + N k J k + 1 = J k χ ^ k | ξ k + W k ,
where λ t and W t are the Gaussian noise with covariance R and Q and zeros mean and ξ is the vector of parameters of T2-FLS that includes rule parameters:
ξ = θ 1 , , θ M T .
The learning algorithm is presented as follows.
(1)
Consider error covariance as ϕ k 1 at sample time k 1 and compute cubature points C h , h = 1 , , 6 M as:
C h , k 1 = ϕ k 1 ς h + ξ k 1 ,
where, M is the number of rules and ς h is:
ς h = 3 M 0 1 h t h e l e m e n t 0 T h = 1 , 2 , , 3 M 3 M 0 1 h t h e l e m e n t 0 T h = 3 M + 1 , , 6 M .
(2)
For each w ι in (16), evaluate the cost function J as:
J h = 1 N i = 1 N χ ^ ¨ i t | C h + k = 1 n χ ^ ˙ i γ k t | C h / P k , i t + χ ^ i t | C h / G i F t 2 + 1 2 χ ^ 0 2 t | C h + χ ^ ˙ 0 2 t | C h ,
where, h = 1 , , 6 M .
(3)
From (18), estimate J m as the mean of J h , h = 1 , , 6 M :
J m = h = 1 6 M J h / 6 M .
(4)
Define Z k 1 as:
Z k 1 = 1 6 M J 1 , k 1 J m , k 1 , J 2 , k 1 J m , k 1 , , J 6 M , k 1 J m , k 1 T .
(5)
From (20), compute the square-root of covariance matrix as:
ϕ z z , k 1 = Tria Z k 1 S R , k ,
where Tria · represents triangularization and S R , k 1 is the square root of R k 1 .
(6)
Compute cross-covariance π ξ z , k 1 as:
π ξ z , k 1 = ζ k 1 Z k 1 T ,
where
ζ k 1 = 1 6 M C 1 , k 1 ξ k 1 , C 2 , k 1 ξ k 1 , , C 6 M , k 1 ξ k 1 .
(7)
Obtain Kalman gain as:
κ k = π ξ z , k 1 / ϕ z z , k 1 T / ϕ z z , k 1 T .
(8)
Update ξ as:
ξ k = ξ k 1 κ k J m .
(9)
Update error covariance as:
S R , k = Tria ζ k 1 κ k Z k 1 κ k S R , k 1 .

5. Stability and Convergence Analysis

To prove the stability and convergence of the suggested algorithm, the Lyapunov approach [40,41] is used. To apply Lyapunov approach, the following Lyapunov function is defined:
V t = 1 N i = 1 N χ ^ ¨ i t + k = 1 n χ ^ i γ k t / P k , i t + χ ^ i t / G i F t 2 .
Time difference of V, results in:
V t V t 1 = 1 N i = 1 N χ ^ ¨ i t + k = 1 n χ ^ i γ k t / P k , i t + χ ^ i t / G i t F t 2 1 N i = 1 N χ ^ ¨ i t 1 + k = 1 n χ ^ i γ k t 1 / P k , i t 1 + χ ^ i t 1 / G i t 1 F t 1 2 .
Considering small sample time, e q . d v 1 can be simplified as:
V t V t 1 1 N i = 1 N χ ^ ¨ i t χ ^ ¨ i t 1 + k = 1 n χ ^ i γ k t χ ^ i γ k t 1 / P k , i t + χ ^ i t χ ^ i t 1 / G i t .
From (13), χ ^ ¨ i t χ ^ ¨ i t 1 is obtained as:
χ ^ ¨ i t χ ^ ¨ i t 1 = l = 1 M θ l t θ l t 1 · 2 t ν l 2 + f ˙ l t l = 1 M 2 t m l ν l 2 + f l t l = 1 M 2 t ν l 2 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M 2 t m l ν l 2 / l = 1 M f l t .
From (11), χ ^ ˙ i γ k t χ ^ i γ k t 1 is written as:
χ ^ ˙ i γ k t χ ^ i γ k t 1 = l = 1 M θ l t θ l t 1 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M f l t .
From (7), χ ^ i t χ ^ i t 1 is computed as:
χ ^ i t χ ^ i t 1 = l = 1 M θ l t θ l t 1 ψ l t , m , ν .
From (29) and Equations (31) and (32), one has:
V t V t 1 = 1 N i = 1 N l = 1 M θ l t θ l t 1 · 2 t ν l 2 + f ˙ l t l = 1 M 2 t m l ν l 2 + f l t l = 1 M 2 t ν l 2 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M 2 t m l ν l 2 / l = 1 M f l t + k = 1 n 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M f l t / P k , i t + ψ l t , m , ν / G i t .
From (25) and (33), one has:
V t V t 1 1 N i = 1 N l = 1 M κ J m · 2 t ν l 2 + f ˙ l t l = 1 M 2 t m l ν l 2 + f l t l = 1 M 2 t ν l 2 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M 2 t m l ν l 2 / l = 1 M f l t + k = 1 n 2 t m l ν l 2 + f l t l = 1 M 2 t m l ν l 2 l = 1 M f l t / P k , i t + ψ l t , m , ν / G i t .
From the fact that J m > 0 , it is concluded that V t V t 1 0 and from the Lyapunov theorem, the stability and boundedness of the cost function is derived.

6. Evaluation Index

To evaluate the accuracy and robustness of the suggested algorithm, the following indexes are defined.
RMSE = 1 N i = 1 N χ i χ ^ i 2 ,
TIC = 1 N i = 1 N χ i χ ^ i 2 / 1 N i = 1 N χ i 2 + 1 N i = 1 N χ ^ i 2 ,
VAR = i = 1 N χ χ ^ i 2 / N 1 ,
where N is the number of sample times, χ i and χ ^ i are the exact and estimated solutions and RMSE , TIC and VAR are root mean square error, inequality coefficient of Theil index, and variance, respectively.

7. Simulations

By several statistical analyses, the accuracy of the suggested algorithm is examined.
Example 1.
For the first examination, an SMDE is considered as:
χ ¨ t + 1 t χ ˙ t / 2 + 1 t 2 χ ˙ t / 4 + 1 1 t χ t = F t , 0 < t 1 ,
where
F t = 1 t α cos α t / 2 + 1 t 2 α cos α t / 4 + α 2 t + 1 α 2 1 t sin α t ,
where, α = π . The real solution of (38), is sin α t . To estimate the solution by the suggested method, the cost function is given as:
J = 1 N u = 0 N 1 u u 2 χ ^ ¨ u | ξ + 1 u u χ ^ ˙ u 2 | ξ + 1 u χ ^ ˙ u 4 | ξ 1 u u 2 F u 2 N = 1 / τ , τ = 0.05 .
The time range 0 , 1 is divided into 21 sections and then we have 21 rules. The standard division of each MF is considered to be 0.1. The trajectories of the output of T2-FLS (approximated solution), mean of approximated solutions and exact solution are depicted in Figure 3a and the corresponding rule parameters are shown in Figure 3b. The absolute error is depicted in Figure 4a and the values of FIT, RMSE, VAR and TIC are shown in Figure 4b. The statistical analyses for TIC, RMSE, VAR and FIT are given in Figure 5, Figure 6, Figure 7 and Figure 8. One can observe that the metrics of TIC, RMSE, VAR and FIT are in the favorable level and the trajectory of the approximated solution well tracks the exact solution x ( t ) = sin ( α t ) .
For the accuracy of the suggested approach to be well seen, the values of interquartile range (IR), median (Med), minimum (Min) and mean of absolute error at each sample time are provided in Table 1. One can see that the values of mean and IR items are in range of 10 3 to 10 2 that indicate an accurate and robust solution.
Example 2.
In this Example, the following SMDE is considered:
χ ¨ t + 1 t χ ˙ t / 2 + 1 t 2 χ ˙ t / 4 + 1 1 t χ t = F t , 0 < t 1 ,
where
F t = 1 t exp t / 2 + 1 t 2 exp t / 4 + 2 t 1 t exp t .
The cost function is:
J = 1 N u = 0 N 1 u u 2 χ ^ ¨ u | w + 1 u u χ ^ ˙ u 2 | w + 1 u χ ^ ˙ u 4 | w 1 u u 2 F u 2 + 1 2 χ ^ 0 | w 1 2 + χ ^ ˙ 0 | w 2 N = 1 / τ , τ = 0.05 .
Similar to Example 1, we have 21 rules. The trajectories of the output of T2-FLS (approximated solution), the mean of approximated solutions and exact solution are depicted in Figure 9a and the corresponding rule parameters are shown in Figure 9b. The absolute error is depicted in Figure 10a and the values of FIT, RMSE, VAR and TIC are shown in Figure 10b. The statistical analysis for TIC, RMSE, VAR and FIT are given in Figure 11, Figure 12, Figure 13 and Figure 14. One can observe that the metrics of TIC, RMSE, VAR and FIT are in the favorable level and the trajectory of the approximated solution well tracks the exact solution x ( t ) = exp ( t ) .
Similar to Example 1, in order for the accuracy of the suggested approach to be well seen, the values of interquartile range (IR), median (Med), minimum (Min) and mean of absolute error at each sample time are provided in Table 2. One can see that the values of mean and IR items are in the range of 10 3 to 10 2 that indicate an accurate and robust solution.
Remark 1.
The main hyperparameters of the suggested algorithm are the number of rules and initial covariance matrices. For stability considerations, the initial covariance matrices are chosen to be relatively small. The number of rules is equal to the number of membership functions for input t. To determine the number of rules, the time range t 0 , T is divided into M sections and for each section one Gaussian membership function with a certain center and an uncertain standard division is considered. The number of rules can affect the accuracy but it depends on the size of sample time. For a smaller sample time, more rules can be taken into account.
Example 3.
In this section, the performance of the suggested algorithm is compared with other similar techniques in the literature [29]. In Reference [29], a simple NN is learned by genetic algorithm to find a solution for a pantograph system. The following MDE is taken into account [29]:
χ ˙ = 1 2 e t / 2 χ t / 2 + 1 2 χ t , χ 0 = 1 .
The exact solution of (44) is χ t = e t . The number of rules in T2-FLS is considered to be 10, the same as the number of neurons in Reference [29]. The numerical comparison, with the method of Reference [29], is given in Table 3. Although the results are close to each other in this Example, the main advantage of the suggested method is that the results of the suggested method are obtained in only one epoch. However, in the genetic algorithm presented in Reference [29], the learning process of NN is repeated several times until a reasonable result is achieved. The evolutionary based learning techniques are not suitable for online applications because of the high level of computational cost and lack of stability guarantee.

8. Conclusions

In this paper, a new approach on the basis of fuzzy neural networks and SCKF is introduced for finding a numerical solution for multi-pantograph singular differential equations. The proposed learning method is stable and this property is shown by a new approach on the basis of the Lypunov theorem. Two simulations are provided to demonstrate the efficiency of the designed solver. Several statistical analyses are given to verify the effectiveness of the introduced algorithm such as the analysis of RMSE, Interquartile Range, Theil’s Inequality Index and Variance metrics. The metrics of TIC, RMSE, VAR and FIT are shown in the favorable level and the trajectory approximated solution well tracks the exact solution. Also, the performance of the suggested method is compared with the other similar techniques in the literature. It is shown that the proposed technique results in better accuracy despite less computational cost in contrast to the evolutionary based learning genetic algorithm.

Author Contributions

The authors contributed to this work as follows. A.M. (Amirhosein Mosavi), M.S., S.S.B. and A.M. (Ardashir Mohammadzadeh) contributed in writing—original draft, formal analysis, investigation, software and methodology. Z.M. and S.N.Q. contributed in writing—review, formal analysis, investigation and methodology. All authors have read and agreed to the published version of the manuscript.

Funding

We acknowledge the support of the German Research Foundation (DFG) and the Bauhaus-Universität Weimar within the Open-Access Publishing Programme.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

SCKFSquare root cubature Kalman filter
SMDESingular multi-pantograph differential equations
T2-FLSType-2 fuzzy logic system
FLSFuzzy logic system
NNNeural network
MDEMulti-pantograph differential equation
RMSERoot mean square error
TICInequality coefficient of Theil index
VARvariance
FITFitness
IRInterquartile range
MedMedian
MinMinimum
MNumber of rules
m l Center of l-th Gaussian membership function
ν ¯ l Upper standard division
ν ̲ l Lower standard division
JCost function
ψ l l-th rule firing
ϕ Covariance matrix
κ Kalman gain
NNumber of data samples

References

  1. Ahmad, I.; Mukhtar, A. Stochastic approach for the solution of multi-pantograph differential equation arising in cell-growth model. Appl. Math. Comput. 2015, 261, 360–372. [Google Scholar]
  2. Heydari, M.; Loghmani, G.; Hosseini, S. Operational matrices of Chebyshev cardinal functions and their application for solving delay differential equations arising in electrodynamics with error estimation. Appl. Math. Model. 2013, 37, 7789–7809. [Google Scholar] [CrossRef]
  3. Hoseini, S.M. Optimal control of linear pantograph-type delay systems via composite Legendre method. J. Frankl. Inst. 2020, 357, 5402–5427. [Google Scholar] [CrossRef]
  4. Ahmad, I.; Raja, M.A.Z.; Bilal, M.; Ashraf, F. Bio-inspired computational heuristics to study Lane–Emden systems arising in astrophysics model. SpringerPlus 2016, 5, 1866. [Google Scholar] [CrossRef] [Green Version]
  5. Ahmad, I.; Ahmad, F.; Bilal, M. Neuro-Heuristic Computational Intelligence for nonlinear Thomas-Fermi equation using trigonometric and hyperbolic approximation. Measurement 2020, 156, 107549. [Google Scholar] [CrossRef]
  6. Ndolane, S. Solutions for some conformable differential equations. Progr. Fract. Differ. Appl. 2018, 4, 493–501. [Google Scholar]
  7. El-Ajou, A.; Moa’ath, N.O.; Al-Zhour, Z.; Momani, S. Analytical numerical solutions of the fractional multi-pantograph system: Two attractive methods and comparisons. Results Phys. 2019, 14, 102500. [Google Scholar] [CrossRef]
  8. Ahmad, E.l.; Omar, A.A.; Shaher, M. Homotopy analysis method for second-order boundary value problems of integrodifferential equations. Discret. Dyn. Nat. Soc. 2012, 2012, 107549. [Google Scholar]
  9. Ndolane, S.; Niang, F.A. Homotopy perturbation ρ-laplace transform method and its application to the fractional diffusion equation and the fractional diffusion-reaction equation. Fractal Fract. 2019, 3, 14. [Google Scholar]
  10. Ezz-Eldien, S.; Wang, Y.; Abdelkawy, M.; Zaky, M.; Aldraiweesh, A.; Machado, J.T. Chebyshev spectral methods for multi-order fractional neutral pantograph equations. Nonlinear Dyn. 2020, 100, 3785–3797. [Google Scholar] [CrossRef]
  11. Vichitkunakorn, P.; Vo, T.N.; Razzaghi, M. A numerical method for fractional pantograph differential equations based on Taylor wavelets. Trans. Inst. Meas. Control 2020, 42, 1334–1344. [Google Scholar] [CrossRef]
  12. Dehestani, H.; Ordokhani, Y.; Razzaghi, M. Numerical technique for solving fractional generalized pantograph-delay differential equations by using fractional-order hybrid bessel functions. Int. J. Appl. Comput. Math. 2020, 6, 1–27. [Google Scholar] [CrossRef]
  13. Sher, M.; Shah, K.; Fečkan, M.; Khan, R.A. Qualitative analysis of multi-terms fractional order delay differential equations via the topological degree theory. Mathematics 2020, 8, 218. [Google Scholar] [CrossRef] [Green Version]
  14. Eriqat, T.; El-Ajou, A.; Moa’ath, N.O.; Al-Zhour, Z.; Momani, S. A New Attractive Analytic Approach for Solutions of Linear and Nonlinear Neutral Fractional Pantograph Equations. Chaos Solitons Fractals 2020, 138, 109957. [Google Scholar] [CrossRef]
  15. Valizadeh, M.; Mahmoudi, Y.; Dastmalchi Saei, F. Application of Natural Transform Method to Fractional Pantograph Delay Differential Equations. J. Math. 2019, 2019, 3913840. [Google Scholar] [CrossRef] [Green Version]
  16. Ebrahimi, H.; Sadri, K. An operational approach for solving fractional pantograph differential equation. Iran. J. Numer. Anal. Optim. 2019, 9, 37–68. [Google Scholar]
  17. Davaeifar, S.; Rashidinia, J. Solution of a system of delay differential equations of multi pantograph type. J. Taibah Univ. Sci. 2017, 11, 1141–1157. [Google Scholar] [CrossRef] [Green Version]
  18. Ezz-Eldien, S.S. On solving systems of multi-pantograph equations via spectral tau method. Appl. Math. Comput. 2018, 321, 63–73. [Google Scholar]
  19. Hafez, R.; Youssri, Y.H. Shifted gegenbauer-gauss collocation method forsolving fractional neutralfunctional-differential equations withproportional delays. Kragujev. J. Math. 2020, 46, 981–996. [Google Scholar]
  20. Youssri, H.; Hafez, R.M. Exponential Jacobi spectral method for hyperbolic partial differential equations. Math. Sci. 2019, 13, 347–354. [Google Scholar] [CrossRef] [Green Version]
  21. Doha, E.H.; Hafez, R.M.; Youssri, Y.H. Shifted Jacobi spectral-Galerkin method for solving hyperbolic partial differential equations. Comput. Math. Appl. 2019, 78, 889–904. [Google Scholar]
  22. Dai, S. Fuzzy Kolmogorov Complexity Based on a Classical Description. Entropy 2020, 22, 66. [Google Scholar] [CrossRef] [Green Version]
  23. Jin, Y.; Ashraf, S.; Abdullah, S. Spherical fuzzy logarithmic aggregation operators based on entropy and their application in decision support systems. Entropy 2019, 21, 628. [Google Scholar] [CrossRef] [Green Version]
  24. Salem, O.A.; Liu, F.; Chen, Y.P.P.; Chen, X. Ensemble Fuzzy Feature Selection Based on Relevancy, Redundancy, and Dependency Criteria. Entropy 2020, 22, 757. [Google Scholar] [CrossRef]
  25. Troussas, C.; Krouska, A.; Sgouropoulou, C.; Voyiatzis, I. Ensemble Learning Using Fuzzy Weights to Improve Learning Style Identification for Adapted Instructional Routines. Entropy 2020, 22, 735. [Google Scholar] [CrossRef]
  26. Batool, B.; Ahmad, M.; Abdullah, S.; Ashraf, S.; Chinram, R. Entropy Based Pythagorean Probabilistic Hesitant Fuzzy Decision Making Technique and Its Application for Fog-Haze Factor Assessment Problem. Entropy 2020, 22, 318. [Google Scholar] [CrossRef] [Green Version]
  27. Yan, B.; Yu, L.; Wang, J. Research on Evaluating the Sustainable Operation of Rail Transit System Based on QFD and Fuzzy Clustering. Entropy 2020, 22, 750. [Google Scholar] [CrossRef]
  28. Ma, L.V.; Park, J.; Nam, J.; Ryu, H.; Kim, J. A fuzzy-based adaptive streaming algorithm for reducing entropy rate of dash bitrate fluctuation to improve mobile quality of service. Entropy 2017, 19, 477. [Google Scholar] [CrossRef] [Green Version]
  29. Raja, M.A.Z.; Ahmad, I.; Khan, I.; Syam, M.I.; Wazwaz, A.M. Neuro-heuristic computational intelligence for solving nonlinear pantograph systems. Front. Inf. Technol. Electron. Eng. 2017, 18, 464–484. [Google Scholar] [CrossRef]
  30. Effati, S.; Pakdaman, M. Artificial neural network approach for solving fuzzy differential equations. Inf. Sci. 2010, 180, 1434–1457. [Google Scholar] [CrossRef]
  31. Hou, C.C.; Simos, T.E.; Famelis, I.T. Neural network solution of pantograph type differential equations. Math. Methods Appl. Sci. 2020, 43, 3369–3374. [Google Scholar] [CrossRef]
  32. Abbasi, A.A.; Javed, S.; Shamshirband, S. An intelligent memory caching architecture for data-intensive multimedia applications. In Multimedia Tools and Applications; Springer: Berlin/Heidelberg, Germany, 2020; pp. 1–19. [Google Scholar]
  33. Inayat, I.; Salim, S.S.; Marczak, S.; Daneva, M.; Shamshirband, S. A systematic literature review on agile requirements engineering practices and challenges. Comput. Hum. Behav. 2015, 51, 915–929. [Google Scholar] [CrossRef]
  34. Shahid, F.; Ashraf, H.; Ghani, A.; Ghayyur, S.A.K.; Shamshirband, S.; Salwana, E. PSDS–Proficient Security Over Distributed Storage: A Method for Data Transmission in Cloud. IEEE Access 2020, 8, 118285–118298. [Google Scholar] [CrossRef]
  35. Samadianfard, S.; Jarhan, S.; Salwana, E.; Mosavi, A.; Shamshirband, S.; Akib, S. Support vector regression integrated with fruit fly optimization algorithm for river flow forecasting in Lake Urmia Basin. Water 2019, 11, 1934. [Google Scholar] [CrossRef] [Green Version]
  36. Nabipour, M.; Nayyeri, P.; Jabani, H.; Mosavi, A.; Salwana, E. Deep learning for Stock Market Prediction. Entropy 2020, 22, 840. [Google Scholar] [CrossRef]
  37. Riahi-Madvar, H.; Dehghani, M.; Seifi, A.; Salwana, E.; Shamshirband, S.; Mosavi, A.; Chau, K.W. Comparative analysis of soft computing techniques RBF, MLP, and ANFIS with MLR and MNLR for predicting grade-control scour hole geometry. Eng. Appl. Comput. Fluid Mech. 2019, 13, 529–550. [Google Scholar] [CrossRef] [Green Version]
  38. Homaei, M.H.; Salwana, E.; Shamshirband, S. An enhanced distributed data aggregation method in the Internet of Things. Sensors 2019, 19, 3173. [Google Scholar] [CrossRef] [Green Version]
  39. Nie, M.; Tan, W.W. Towards an efficient type-reduction method for interval type-2 fuzzy logic systems. In Proceedings of the 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence), Hong Kong, China, 1–6 June 2008; pp. 1425–1432. [Google Scholar]
  40. Khalil, H.K. Lyapunov Stability. Available online: https://www.eolss.net/Sample-Chapters/C18/E6-43-21-05.pdf (accessed on 13 August 2020).
  41. Ndolane, S. Exponential form for Lyapunov function and stability analysis of the fractional differential equations. J. Math. Comput. Sci 2018, 18, 388–397. [Google Scholar]
Figure 1. Block diagram of the proposed solver.
Figure 1. Block diagram of the proposed solver.
Entropy 22 01041 g001
Figure 2. The structure of suggested T2-FLS.
Figure 2. The structure of suggested T2-FLS.
Entropy 22 01041 g002
Figure 3. Example 1: (a): Solution performance; (b): Weights of NN.
Figure 3. Example 1: (a): Solution performance; (b): Weights of NN.
Entropy 22 01041 g003
Figure 4. Example 1: (a): Absolute error; (b): The values of FIT, RMSE, VAR and TIC.
Figure 4. Example 1: (a): Absolute error; (b): The values of FIT, RMSE, VAR and TIC.
Entropy 22 01041 g004
Figure 5. Example 1: (a): The value of TIC at each iteration; (b): Histogram plot for TIC; (c): Box plot for TIC.
Figure 5. Example 1: (a): The value of TIC at each iteration; (b): Histogram plot for TIC; (c): Box plot for TIC.
Entropy 22 01041 g005
Figure 6. Example 1: (a): The value of RMSE at each iteration; (b): Histogram plot for RMSE; (c): Box plot for RMSE.
Figure 6. Example 1: (a): The value of RMSE at each iteration; (b): Histogram plot for RMSE; (c): Box plot for RMSE.
Entropy 22 01041 g006
Figure 7. Example 1: (a): The value of VAR at each iteration; (b): Histogram plot for VAR; (c): Box plot for VAR.
Figure 7. Example 1: (a): The value of VAR at each iteration; (b): Histogram plot for VAR; (c): Box plot for VAR.
Entropy 22 01041 g007
Figure 8. Example 1: (a): The value of FIT at each iteration; (b): Histogram plot for FIT; (c): Box plot for FIT.
Figure 8. Example 1: (a): The value of FIT at each iteration; (b): Histogram plot for FIT; (c): Box plot for FIT.
Entropy 22 01041 g008
Figure 9. Example 2: (a): Solution performance; (b): Weights of NN.
Figure 9. Example 2: (a): Solution performance; (b): Weights of NN.
Entropy 22 01041 g009
Figure 10. Example 2: (a): Absolute error; (b): The values of FIT, RMSE, VAR and TIC.
Figure 10. Example 2: (a): Absolute error; (b): The values of FIT, RMSE, VAR and TIC.
Entropy 22 01041 g010
Figure 11. Example 2: (a): The value of TIC at each iteration; (b): Histogram plot for TIC; (c): Box plot for TIC.
Figure 11. Example 2: (a): The value of TIC at each iteration; (b): Histogram plot for TIC; (c): Box plot for TIC.
Entropy 22 01041 g011
Figure 12. Example 2: (a): The value of RMSE at each iteration; (b): Histogram plot for RMSE; (c): Box plot for RMSE.
Figure 12. Example 2: (a): The value of RMSE at each iteration; (b): Histogram plot for RMSE; (c): Box plot for RMSE.
Entropy 22 01041 g012
Figure 13. Example 2: (a): The value of VAR at each iteration; (b): Histogram plot for VAR; (c): Box plot for VAR.
Figure 13. Example 2: (a): The value of VAR at each iteration; (b): Histogram plot for VAR; (c): Box plot for VAR.
Entropy 22 01041 g013
Figure 14. Example 2: (a): The value of FIT at each iteration; (b): Histogram plot for FIT; (c): Box plot for FIT.
Figure 14. Example 2: (a): The value of FIT at each iteration; (b): Histogram plot for FIT; (c): Box plot for FIT.
Entropy 22 01041 g014
Table 1. Example 1: Statistical analysis.
Table 1. Example 1: Statistical analysis.
tMinMeanMedIR
00.02180.07760.08230.0451
0.05000.02280.07860.08300.0448
0.10000.02230.07880.08350.0455
0.15000.02000.07740.08270.0473
0.20000.01540.07360.07990.0498
0.25000.00890.06720.07460.0519
0.30000.00140.05890.06750.0523
0.35000.00400.05050.05950.0520
0.40000.00010.04300.05140.0500
0.45000.00300.03730.04340.0416
0.50000.00390.03230.03540.0328
0.55000.00030.02790.02780.0202
0.60000.00060.02420.02310.0139
0.65000.00420.02110.02090.0129
0.70000.00260.01790.01670.0159
0.75000.00010.01490.01240.0138
0.80000.00190.01210.01200.0127
0.85000.00120.00920.00910.0112
0.90000.00000.00620.00570.0087
0.95000.00030.00330.00300.0053
1.00000.00000.00000.00000.0000
Table 2. Example 2: Statistical analysis.
Table 2. Example 2: Statistical analysis.
tMinMeanMedIR
00.00040.00060.00060.0001
0.05000.00060.00090.00090.0002
0.10000.00100.00140.00150.0003
0.15000.00130.00160.00160.0002
0.20000.00000.00080.00080.0005
0.25000.00020.00140.00130.0014
0.30000.00130.00480.00470.0025
0.35000.00380.00900.00920.0038
0.40000.00650.01330.01370.0050
0.45000.00900.01710.01760.0059
0.50000.01120.02040.02100.0067
0.55000.01340.02340.02400.0073
0.60000.01550.02610.02680.0077
0.65000.01750.02850.02920.0079
0.70000.01910.03010.03090.0079
0.75000.02010.03070.03160.0075
0.80000.02060.03060.03140.0069
0.85000.02070.02980.03060.0064
0.90000.02050.02830.02910.0057
0.95000.01930.02550.02620.0041
1.00000.01570.02010.02050.0028
Table 3. Example 3: Comparison.
Table 3. Example 3: Comparison.
tExact SolutionProposed MethodMethod of Reference [29]
01.00001.00001.0000
0.11.10521.10511.1051
0.21.22141.22131.2213
0.31.34991.34981.3497
0.41.49181.49171.4917
0.51.64871.64861.6486
0.61.82211.82211.8220
0.72.01382.01372.0136
0.82.22552.22542.2253
0.92.45962.45952.4594
12.71832.71812.7181

Share and Cite

MDPI and ACS Style

Mosavi, A.; Shokri, M.; Mansor, Z.; Qasem, S.N.; Band, S.S.; Mohammadzadeh, A. Machine Learning for Modeling the Singular Multi-Pantograph Equations. Entropy 2020, 22, 1041. https://doi.org/10.3390/e22091041

AMA Style

Mosavi A, Shokri M, Mansor Z, Qasem SN, Band SS, Mohammadzadeh A. Machine Learning for Modeling the Singular Multi-Pantograph Equations. Entropy. 2020; 22(9):1041. https://doi.org/10.3390/e22091041

Chicago/Turabian Style

Mosavi, Amirhosein, Manouchehr Shokri, Zulkefli Mansor, Sultan Noman Qasem, Shahab S. Band, and Ardashir Mohammadzadeh. 2020. "Machine Learning for Modeling the Singular Multi-Pantograph Equations" Entropy 22, no. 9: 1041. https://doi.org/10.3390/e22091041

APA Style

Mosavi, A., Shokri, M., Mansor, Z., Qasem, S. N., Band, S. S., & Mohammadzadeh, A. (2020). Machine Learning for Modeling the Singular Multi-Pantograph Equations. Entropy, 22(9), 1041. https://doi.org/10.3390/e22091041

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop