Nothing Special   »   [go: up one dir, main page]
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Encoding physics to learn reaction–diffusion processes

Nature Machine Intelligence volume 5, pages 765–779 (2023)Cite this article

Subjects

A preprint version of the article is available at arXiv.

Abstract

Modelling complex spatiotemporal dynamical systems, such as reaction–diffusion processes, which can be found in many fundamental dynamical effects in various disciplines, has largely relied on finding the underlying partial differential equations (PDEs). However, predicting the evolution of these systems remains a challenging task for many cases owing to insufficient prior knowledge and a lack of explicit PDE formulation for describing the nonlinear process of the system variables. With recent data-driven approaches, it is possible to learn from measurement data while adding prior physics knowledge. However, existing physics-informed machine learning paradigms impose physics laws through soft penalty constraints, and the solution quality largely depends on a trial-and-error proper setting of hyperparameters. Here we propose a deep learning framework that forcibly encodes a given physics structure in a recurrent convolutional neural network to facilitate learning of the spatiotemporal dynamics in sparse data regimes. We show with extensive numerical experiments how the proposed approach can be applied to a variety of problems regarding reaction–diffusion processes and other PDE systems, including forward and inverse analysis, data-driven modelling and discovery of PDEs. We find that our physics-encoding machine learning approach shows high accuracy, robustness, interpretability and generalizability.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Schematic architecture of the PeRCNN.
Fig. 2: Error propagation curve and predicted snapshots by PeRCNN, ConvLSTM and PINN on various RD systems.
Fig. 3: Snapshots of the measurement data employed in the experiment and the identified coefficients.
Fig. 4: Error propagation curve of the prediction and the extrapolated snapshots from each data-driven model compared with the reference solution.
Fig. 5: Error propagation curve and the snapshots of the inference result.
Fig. 6: Flowchart of the discovery of governing PDEs.

Similar content being viewed by others

Data availability

All the used datasets in this study are available in the Zenodo repository71, the Gitee repository at https://gitee.com/chengzrz/percnn and the GitHub repository at https://github.com/isds-neu/PeRCNN.

Code availability

All the source codes to reproduce the results in this study are available in the Zenodo repository71, the Gitee repository at https://gitee.com/chengzrz/percnn and the GitHub repository at https://github.com/isds-neu/PeRCNN.

References

  1. Raissi, M., Perdikaris, P. & Karniadakis, G. E. Machine learning of linear differential equations using Gaussian processes. J. Comput. Phys. 348, 683–693 (2017).

  2. Han, J., Jentzen, A. & Weinan, E. Solving high-dimensional partial differential equations using deep learning. Proc. Natl Acad. Sci. USA 115, 8505–8510 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  3. Bar-Sinai, Y., Hoyer, S., Hickey, J. & Brenner, M. P. Learning data-driven discretizations for partial differential equations. Proc. Natl Acad. Sci. USA 116, 15344–15349 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  4. Sanchez-Gonzalez, A. et al. Learning to simulate complex physics with graph networks. In International Conference on Machine Learning 8459–8468 (PMLR, 2020).

  5. Long, Z., Lu, Y., Ma, X. & Dong, B. PDE-Net: learning PDEs from data. In International Conference on Machine Learning 3208–3216 (PMLR, 2018).

  6. Wang, R., Kashinath, K., Mustafa, M., Albert, A. & Yu, R. Towards physics-informed deep learning for turbulent flow prediction. In Proc. 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 1457–1466 (ACM, 2020).

  7. Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A. & Battaglia, P. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations (OpenReview.net, 2021).

  8. de Avila Belbute-Peres, F., Economon, T. & Kolter, Z. Combining differentiable PDE solvers and graph neural networks for fluid flow prediction. In International Conference on Machine Learning 2402–2411 (PMLR, 2020).

  9. Kochkov, D. et al. Machine learning-accelerated computational fluid dynamics. Proc. Natl Acad. Sci. USA 118, e2101784118 (2021).

    Article  MathSciNet  Google Scholar 

  10. Erichson, N. B. Shallow neural networks for fluid flow reconstruction with limited sensors. Proc. R. Soc. A 476, 20200097 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  11. Stengel, K., Glaws, A., Hettinger, D. & King, R. N. Adversarial super-resolution of climatological wind and solar data. Proc. Natl Acad. Sci. USA 117, 16805–16815 (2020).

    Article  Google Scholar 

  12. Fukami, K., Fukagata, K. & Taira, K. Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows. J. Fluid Mech. 909, A9 (2021).

  13. Rao, C. & Liu, Y. Three-dimensional convolutional neural network (3D-CNN) for heterogeneous material homogenization. Comput. Mater. Sci. 184, 109850 (2020).

    Article  Google Scholar 

  14. Schmidt, M. & Lipson, H. Distilling free-form natural laws from experimental data. Science 324, 81–85 (2009).

    Article  Google Scholar 

  15. Brunton, S. L., Proctor, J. L. & Kutz, J. N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 113, 3932–3937 (2016).

    Article  MathSciNet  MATH  Google Scholar 

  16. Rudy, S. H., Brunton, S. L., Proctor, J. L. & Kutz, J. N. Data-driven discovery of partial differential equations. Sci. Adv. 3, e1602614 (2017).

    Article  Google Scholar 

  17. Udrescu, S.-M. & Tegmark, M. AI Feynman: a physics-inspired method for symbolic regression. Sci. Adv. 6, eaay2631 (2020).

  18. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  19. Lusch, B., Kutz, J. N. & Brunton, S. L. Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun. 9, 4950 (2018).

    Article  Google Scholar 

  20. Long, Z., Lu, Y. & Dong, B. PDE-Net 2.0: learning PDEs from data with a numeric-symbolic hybrid deep network. J. Comput. Phys. 399, 108925 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  21. Chen, Z., Liu, Y. & Sun, H. Physics-informed learning of governing equations from scarce data. Nat. Commun. 12, 6136 (2021).

    Article  Google Scholar 

  22. Cranmer, M. D. et al. Discovering symbolic models from deep learning with inductive biases. In Advances in Neural Information Processing Systems (Curran Associates, 2020).

  23. Karpatne, A. et al. Theory-guided data science: a new paradigm for scientific discovery from data. IEEE Trans. Knowl. Data Eng. 29, 2318–2331 (2017).

    Article  Google Scholar 

  24. Raissi, M., Perdikaris, P. & Karniadakis, G. E. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  25. Raissi, M., Yazdani, A. & Karniadakis, G. E. Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations. Science 367, 1026–1030 (2020).

  26. Karniadakis, G. E. et al. Physics-informed machine learning. Nat. Rev. Phys. 3, 422–440 (2021).

  27. Rao, C., Sun, H. & Liu, Y. Physics-informed deep learning for incompressible laminar flows. Theor. Appl. Mech. Lett. 10, 207–212 (2020).

    Article  Google Scholar 

  28. Sheng, H. & Yang, C. PFNN: a penalty-free neural network method for solving a class of second-order boundary-value problems on complex geometries. J. Comput. Phys. 428, 110085 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  29. Sun, L., Gao, H., Pan, S. & Wang, J.-X. Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data. Comput. Methods Appl. Mech. Eng. 361, 112732 (2020).

  30. Kim, Y., Choi, Y., Widemann, D. & Zohdi, T. A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder. J. Comput. Phys. 451, 110841 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  31. Yang, Y. & Perdikaris, P. Adversarial uncertainty quantification in physics-informed neural networks. J. Comput. Phys. 394, 136–152 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  32. Zhu, Y., Zabaras, N., Koutsourelakis, P.-S. & Perdikaris, P. Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. J. Comput. Phys. 394, 56–81 (2019).

  33. Haghighat, E., Raissi, M., Moure, A., Gomez, H. & Juanes, R. A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics. Comput. Methods Appl. Mech. Eng. 379, 113741 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  34. Jin, X., Cai, S., Li, H. & Em Karniadakis, G. NSFnets (Navier–Stokes flow nets): physics-informed neural networks for the incompressible Navier–Stokes equations. J. Comput. Phys. 426, 109951 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  35. He, Q. Z., Barajas-Solano, D., Tartakovsky, G. & Tartakovsky, A. M. Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport. Adv. Water Resour. 141, 103610 (2020).

  36. He, Q. Z. & Tartakovsky, A. M. Physics-informed neural network method for forward and backward advection-dispersion equations. Water Resour. Res. 57, e2020WR029479 (2021).

  37. Zhang, R., Liu, Y. & Sun, H. Physics-informed multi-lstm networks for metamodeling of nonlinear structures. Comput. Methods Appl. Mech. Eng. 369, 113226 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  38. Rao, C., Sun, H. & Liu, Y. Physics-informed deep learning for computational elastodynamics without labeled data. J. Eng. Mech. 147, 04021043 (2021).

    Article  Google Scholar 

  39. Niaki, S. A., Haghighat, E., Campbell, T., Poursartip, A. & Vaziri, R. Physics-informed neural network for modelling the thermochemical curing process of composite-tool systems during manufacture. Comput. Methods Appl. Mech. Eng. 384, 113959 (2021).

  40. Weinan, E. & Yu, B. The Deep Ritz Method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6, 1–12 (2018).

  41. Ren, P., Rao, C., Liu, Y., Wang, J.-X. & Sun, H. PhyCRNet: physics-informed convolutional-recurrent network for solving spatiotemporal PDEs. Compu. Methods Appl. Mech. Eng. 389, 114399 (2022).

  42. Gao, H., Sun, L. & Wang, J.-X. PhyGeoNet: physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain. J. Comput. Phys. 428, 110079 (2021).

  43. Gao, H., Zahr, M. J. & Wang, J.-X. Physics-informed graph neural galerkin networks: a unified framework for solving PDE-governed forward and inverse problems. Comput. Methods Appl. Mech. Eng. 390, 114502 (2022).

  44. Geneva, N. & Zabaras, N. Transformers for modeling physical systems. Neural Netw. 146, 272–289 (2021).

  45. Geneva, N. & Zabaras, N. Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks. J. Comput. Phys. 403, 109056 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  46. Gao, H., Sun, L. & Wang, J.-X. Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels. Phys. Fluids 33, 073603 (2021).

  47. Lu, L., Jin, P., Pang, G., Zhang, Z. & Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat. Mach. Intell. 3, 218–229 (2021).

    Article  Google Scholar 

  48. Li, Z. et al. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations (OpenReview.net, 2021).

  49. Wang, S., Wang, H. & Perdikaris, P. Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Sci. Adv. 7, eabi8605 (2021).

    Article  Google Scholar 

  50. Halatek, J. & Frey, E. Rethinking pattern formation in reaction–diffusion systems. Nat. Phys. 14, 507–514 (2018).

    Article  Google Scholar 

  51. Holmes, E. E., Lewis, M. A., Banks, J. E. & Veit, R. R. Partial differential equations in ecology: spatial interactions and population dynamics. Ecology 75, 17–29 (1994).

    Article  Google Scholar 

  52. Vervloet, D., Kapteijn, F., Nijenhuis, J. & van Ommen, J. R. Fischer–Tropsch reaction–diffusion in a cobalt catalyst particle: aspects of activity and selectivity for a variable chain growth probability. Catal. Sci. Technol. 2, 1221–1233 (2012).

    Article  Google Scholar 

  53. Maini, P. K., McElwain, D. L. S. & Leavesley, D. I. Traveling wave model to interpret a wound-healing cell migration assay for human peritoneal mesothelial cells. Tissue Eng. 10, 475–482 (2004).

  54. Shi, X. et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In Advances in Neural Information Processing Systems 802–810 (Curran Associates, 2015).

  55. Liao, Q. & Poggio, T. Bridging the gaps between residual learning, recurrent neural networks and visual cortex. Preprint at arXiv https://arxiv.org/abs/1604.03640 (2016).

  56. Zhang, J., Zheng, Y. & Qi, D. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Proc. AAAI Conference on Artificial Intelligence Vol. 31, 1655–1661 (AAAI, 2017).

  57. Raissi, M. Deep hidden physics models: deep learning of nonlinear partial differential equations. J. Mach. Learn. Res. 19, 932–955 (2018).

    MathSciNet  MATH  Google Scholar 

  58. Rao, C., Ren, P., Liu, Y. & Sun, H. Discovering nonlinear PDEs from scarce data with physics-encoded learning. In International Conference on Learning Representations (OpenReview.net, 2022).

  59. Meurer, A. SymPy: symbolic computing in Python. PeerJ Comput. Sci. 3, e103 (2017).

    Article  Google Scholar 

  60. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016).

  61. Chen, Y., Yu, W. & Pock, T. On learning optimized reaction diffusion processes for effective image restoration. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 5261–5269 (IEEE, 2015).

  62. Lu, Y., Zhong, A., Li, Q. & Dong, B. Beyond finite layer neural networks: bridging deep architectures and numerical differential equations. In International Conference on Machine Learning 3276–3285 (PMLR, 2018).

  63. Ruthotto, L. & Haber, E. Deep neural networks motivated by partial differential equations. J. Math. Imaging Vis. 62, 352–364 (2019).

  64. Larsson, G., Maire, M. & Shakhnarovich, G. FractalNet: ultra-deep neural networks without residuals. In International Conference on Learning Representations (OpenReview.net, 2017).

  65. Le Guen, V. & Thome, N. Disentangling physical dynamics from unknown factors for unsupervised video prediction. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 11474–11484 (IEEE, 2020).

  66. Cai, J.-F., Dong, B., Osher, S. & Shen, Z. Image restoration: total variation, wavelet frames, and beyond. J. Am. Math. Soc. 25, 1033–1089 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  67. Raissi, M., Wang, Z., Triantafyllou, M. S. & Karniadakis, G. E. Deep learning of vortex-induced vibrations. J. Fluid Mech. 861, 119–137 (2019).

  68. Haupt, J. & Nowak, R. Signal reconstruction from noisy random projections. IEEE Trans. Inf. Theor. 52, 4036–4048 (2006).

    Article  MathSciNet  MATH  Google Scholar 

  69. Blumensath, T. & Davies, M. E. Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27, 265–274 (2009).

    Article  MathSciNet  MATH  Google Scholar 

  70. Tibshirani, R. Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Ser. B 58, 267–288 (1996).

    MathSciNet  MATH  Google Scholar 

  71. isds-neu & Ren, P. isds-neu/PeRCNN: encoding physics to learn reaction-diffusion processes. Zenodo https://doi.org/10.5281/zenodo.7955830 (2023).

Download references

Acknowledgements

The work is supported by the National Natural Science Foundation of China (no. 92270118 and no. 62276269), the Beijing Natural Science Foundation (no. 1232009), the National Key R&D Program of China (no. 2021ZD0110400) and the Beijing Outstanding Young Scientist Program (no. BJJWZYJH012019100020098). We also acknowledge the support by the Huawei MindSpore platform. Y.L. and H.S. acknowledge the support from the Fundamental Research Funds for the Central Universities. C.R. acknowledges the sponsorship of visiting research by H.S. at Renmin University of China.

Author information

Author notes

  1. These authors contributed equally: Chengping Rao, Pu Ren.

Authors and Affiliations

  1. Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China

    Chengping Rao, Qi Wang & Hao Sun

  2. Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA, USA

    Chengping Rao

  3. Department of Civil and Environmental Engineering, Northeastern University, Boston, MA, USA

    Pu Ren

  4. Department of Civil and Environmental Engineering, MIT, Cambridge, MA, USA

    Oral Buyukozturk

  5. Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China

    Hao Sun

  6. School of Engineering Science, University of Chinese Academy of Sciences, Beijing, China

    Yang Liu

Authors
  1. Chengping Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pu Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Oral Buyukozturk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.R., H.S. and Y.L. contributed to the ideation and design of the research. C.R., P.R. and Q.W. performed the research. C.R., P.R., O.B., H.S. and Y.L. wrote the paper.

Corresponding authors

Correspondence to Hao Sun or Yang Liu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks Ilias Bilionis and Lu Lu for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Table 1 Computational parameters for datasets generation
Full size table
Extended Data Table 2 Summary of the coefficient identification results for 2D Gray–Scott reaction–diffusion system
Full size table
Extended Data Table 3 Discovered PDEs from the measurement data under various noise levels compared with the ground truth
Full size table

Supplementary information

Supplementary Information

Supplementary Figs. 1–24 and Tables 1–19.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rao, C., Ren, P., Wang, Q. et al. Encoding physics to learn reaction–diffusion processes. Nat Mach Intell 5, 765–779 (2023). https://doi.org/10.1038/s42256-023-00685-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-023-00685-7

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics