Error estimates for POD-DL-ROMs: a deep learning framework for reduced order modeling of nonlinear parametrized PDEs enhanced by proper orthogonal decomposition

POD-DL-ROMs have been recently proposed as an extremely versatile strategy to build accurate and reliable reduced order models (ROMs) for nonlinear parametrized partial differential equations, combining (i) a preliminary dimensionality reduction obtained through proper orthogonal decomposition (POD) for the sake of efficiency, (ii) an autoencoder architecture that further reduces the dimensionality of the POD space to a handful of latent coordinates, and (iii) a dense neural network to learn the map that describes the dynamics of the latent coordinates as a function of the input parameters and the time variable. Within this work, we aim at justifying the outstanding approximation capabilities of POD-DL-ROMs by means of a thorough error analysis, showing how the sampling required to generate training data, the dimension of the POD space, and the complexity of the underlying neural networks, impact on the solutions us to formulate practical criteria to control the relative error in the approximation of the solution field of interest, and derive general error estimates. Furthermore, we show that, from a theoretical point of view, POD-DL-ROMs outperform several deep learning-based techniques in terms of model complexity. Finally, we validate our findings by means of suitable numerical experiments, ranging from parameter-dependent operators analytically defined to several parametrized PDEs.

The source code implementation of the method described in the paper is made available from the GitHub repository: https://github.com/DLROM-hub/poddlrom-error-estimates together with a sample numerical experiment to showcase a possible use.

The authors are members of the Gruppo Nazionale Calcolo Scientifico-Istituto Nazionale di Alta Matematica (GNCS-INdAM) and acknowledge the project “Dipartimento di Eccellenza” 2023-2027, funded by MUR, as well as the support of Fondazione Cariplo, Italy, Grant n. 2019-4608. AM acknowledges the PRIN 2022 Project “Numerical approximation of un- certainty quantification problems for PDEs by multi-fidelity methods (UQ-FLY)” (No. 202222PACR), funded by the European Union - NextGenerationEU. AM and SF acknowledge the project FAIR (Future Artificial Intelligence Research), funded by the NextGenerationEU program within the PNRR-PE-AI scheme (M4C2, Investment 1.3, Line on Artificial Intelligence). SF also acknowledges the Isaac Newton Institute for Mathematical Sciences, Cambridge, UK, for support and hospitality during the program “The mathematical and statistical foundation of future data-driven engineering,” EPSRC grant no EP/R014604, where part of this work was undertaken.

Open access funding provided by Politecnico di Milano within the CRUI-CARE Agreement.

Authors and Affiliations

1. MOX - Dipartimento di Matematica, Politecnico di Milano, P.zza Leonardo da Vinci 32, I-20133, Milano, Italy

   Simone Brivio, Stefania Fresca, Nicola Rares Franco & Andrea Manzoni

Corresponding author

Correspondence to Andrea Manzoni.

Conflict of interest

The authors declare no competing interests.

Communicated by: Tobias Breiten

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A. Additional proofs

1.1 A.1 Proof of Proposition 1

Thanks to Assumption 1, trivially we obtain \({\varDelta } t = T N_t^{-1} = O(N_t^{-1})\) and we set \(t_i = i{\varDelta } t\). Letting \(f=f(\varvec{\mu },t)\) be the (sufficiently regular) integrand of the integral that we want to approximate, we obtain

where

and

Notice that

because

since \(f \in L^2(\mathcal {P} \times \mathcal {T})\). Thus, the error we commit in approximating the integral goes to zero upon requiring \(N_s,N_t \rightarrow \infty \). Finally, notice that

which allows us to write

1.2 A.2 Proof of Proposition 2

We notice immediately that the integral is well defined \(\forall \textbf{v} \in L^2(\mathcal {P} \times \mathcal {T}; \mathbb {R}^{N_h})\) thanks to the boundedness assumptions on the solution \(\textbf{u} \in L^2(\mathcal {P} \times \mathcal {T}; \mathbb {R}^{N_h})\). We also remark that the boundedness hypotheses may be relaxed: our choice was aimed at consistency with the other theoretical results of the present work. In order to prove that \(\Vert \cdot \Vert _{L^2_w}\) is a norm, we have to show that:

1. (i)

   It satisfies the triangle inequality. Given \(\textbf{v}, \textbf{z} \in L^2(\mathcal {P}\times \mathcal {T}; \mathbb {R}^{N_h})\), by means of the triangular inequality, it is trivial to show that

   $$\begin{aligned} \begin{aligned}&\Vert \textbf{v} + \textbf{z}\Vert ^2_{L^2_w} = \\ {}&= \int _{\mathcal {P} \times \mathcal {T}}\Vert \textbf{v}(\varvec{\mu },t) + \textbf{z}(\varvec{\mu },t)\Vert ^2 w(\varvec{\mu },t) d(\varvec{\mu },t) \le \\ {}&\le \int _{\mathcal {P} \times \mathcal {T}} (\Vert \textbf{v}(\varvec{\mu },t) \Vert + \Vert \textbf{z}(\varvec{\mu },t)\Vert )^2 w(\varvec{\mu },t) d(\varvec{\mu },t) = \\ {}&= \int _{\mathcal {P} \times \mathcal {T}} (\Vert \textbf{v}(\varvec{\mu },t) \Vert ^2 + \Vert \textbf{z}(\varvec{\mu },t)\Vert ^2 + 2\Vert \textbf{v}(\varvec{\mu },t) \Vert \Vert \textbf{z}(\varvec{\mu },t) \Vert )w(\varvec{\mu },t) d(\varvec{\mu },t). \end{aligned} \end{aligned}$$

   Moreover, by the Cauchy-Schwarz inequality, the following inequality holds,

   $$\begin{aligned} \begin{aligned}&\int _{\mathcal {P} \times \mathcal {T}} \Vert \textbf{v}(\varvec{\mu },t) \Vert \Vert \textbf{z}(\varvec{\mu },t) \Vert w(\varvec{\mu },t) d(\varvec{\mu },t) \le \\ {}&\le \sqrt{\int _{\mathcal {P} \times \mathcal {T}} \Vert \textbf{v}(\varvec{\mu },t) \Vert ^2 w(\varvec{\mu },t) d(\varvec{\mu },t) \int _{\mathcal {P} \times \mathcal {T}} \Vert \textbf{z}(\varvec{\mu },t) \Vert ^2 w(\varvec{\mu },t) d(\varvec{\mu },t)}. \end{aligned} \end{aligned}$$

   Thus, we can infer

   $$\begin{aligned} \Vert \textbf{v} + \textbf{z}\Vert ^2_{L^2_w} \le \Vert \textbf{v}\Vert ^2_{L^2_w} + \Vert \textbf{z}\Vert ^2_{L^2_w} + 2\Vert \textbf{v}\Vert _{L^2_w} \Vert \textbf{z}\Vert _{L^2_w}=(\Vert \textbf{v}\Vert _{L^2_w} + \Vert \textbf{z}\Vert _{L^2_w})^2 \end{aligned}$$

   and derive the thesis;

2. (ii)

   \(\Vert \cdot \Vert _{L^2_w}\) is homogeneous thanks to the linearity of the integral;

3. (iii)

   If \(\textbf{v} \in L^2(\mathcal {P} \times \mathcal {T}; \mathbb {R}^{N_h})\), \(\Vert \textbf{v}\Vert _{L^2_w} = 0\) implies that \(\textbf{v} = \varvec{0}\) a.e. by trivial arguments.

1.3 A.3 Quasi-optimality of the discrete formulation of the POD decomposition

We base the following analysis on the results of the \((\mathcal {P} \times \mathcal {T})\)-continuous problem proposed in [36]. We first recall that by definition \(\textbf{V}_\infty \in \mathbb {R}^{N_h \times N}\) (where N is the POD dimension) is optimal for the \((\mathcal {P} \times \mathcal {T})\)-continuous formulation, that is with respect to the \(L^2(\mathcal {P} \times \mathcal {T}; \mathbb {R}^{N_h})\) norm. Formally, we set \(\delta ,\varepsilon > 0\) and, by assuming \(\textbf{u}(\varvec{\mu },t) \in L^2(\mathcal {P} \times \mathcal {T},\mathbb {R}^{N_h})\), we define \(T:L^2(\mathcal {P} \times \mathcal {T}) \rightarrow \mathbb {R}^{N_h}\) as

The adjoint operator of T, namely \(T^*\), enjoys the property

Moreover, recall the definition of the (continuous) correlation matrix () and denote by \((\sigma _{k,\infty }^2,\varvec{\zeta }_k)\) its eigenpairs (where \(\{\varvec{\zeta }_k\}_{k}\) denotes an orthonormal basis). We thus define the HS-norm of T as

Setting

we denote by \(T_{N,\infty }\) the rank-N Schmidt approximation, with

and by \(T_{N} = \textbf{V} \textbf{V}^T T\) its approximation by means of the discrete POD formulation.

where \(\mathcal {B}_N = \{ B \in \mathcal {L}(L^2(\mathcal {P} \times \mathcal {T}); \mathbb {R}^{N_h})): \text {rank}(B) \le N \wedge \Vert B\Vert _{HS} < +\infty \}\), being \(\mathcal {L}(U)\) the space of linear continuous operators from U to U, for U Banach. Now, suppose to define \(B_N \in \mathcal {B}_N\) which does not attain the minimum in (13), thus

By means of the results of Theorem 1, with the same hypotheses, we have that

Thus, since a.s. convergence implies convergence in probability, we derive that

Finally, thanks to (14), we have

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions