NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data

Yifan Sun, Linan Zhang, Hayden Schaeffer

Proceedings of The First Mathematical and Scientific Machine Learning Conference, PMLR 107:352-372, 2020.

Abstract

We propose a neural network based approach for extracting models from dynamic data using ordinary and partial differential equations. In particular, given a time-series or spatio-temporal dataset, we seek to identify an accurate governing system which respects the intrinsic differential structure. The unknown governing model is parameterized by using both (shallow) multilayer perceptrons and nonlinear differential terms, in order to incorporate relevant correlations between spatio-temporal samples. We demonstrate the approach on several examples where the data is sampled from various dynamical systems and give a comparison to recurrent networks and other data-discovery methods. In addition, we show that for SVHN, MNIST, Fashion MNIST, and CIFAR10/100, our approach lowers the parameter cost as compared to other deep neural networks.

Cite this Paper

BibTeX


@InProceedings{pmlr-v107-sun20a,
  title = 	 {{NeuPDE: N}eural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data},
  author =       {Sun, Yifan and Zhang, Linan and Schaeffer, Hayden},
  booktitle = 	 {Proceedings of The First Mathematical and Scientific Machine Learning Conference},
  pages = 	 {352--372},
  year = 	 {2020},
  editor = 	 {Lu, Jianfeng and Ward, Rachel},
  volume = 	 {107},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {20--24 Jul},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v107/sun20a/sun20a.pdf},
  url = 	 {https://proceedings.mlr.press/v107/sun20a.html},
  abstract = 	 {  We propose a neural network based approach for extracting models from dynamic data using ordinary and partial differential equations. In particular, given a time-series or spatio-temporal dataset,  we seek to identify an accurate governing system which respects the intrinsic differential structure. The unknown governing model is parameterized by using both (shallow) multilayer perceptrons and nonlinear differential terms, in order to incorporate relevant correlations between spatio-temporal samples. We demonstrate the approach on several examples where the data is sampled from various dynamical systems and give a comparison to recurrent networks and other data-discovery methods. In addition, we show that for SVHN, MNIST, Fashion MNIST, and CIFAR10/100, our approach lowers the parameter cost as compared to other deep neural networks.}
}

Endnote

%0 Conference Paper
%T NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data
%A Yifan Sun
%A Linan Zhang
%A Hayden Schaeffer
%B Proceedings of The First Mathematical and Scientific Machine Learning Conference
%C Proceedings of Machine Learning Research
%D 2020
%E Jianfeng Lu
%E Rachel Ward	
%F pmlr-v107-sun20a
%I PMLR
%P 352--372
%U https://proceedings.mlr.press/v107/sun20a.html
%V 107
%X   We propose a neural network based approach for extracting models from dynamic data using ordinary and partial differential equations. In particular, given a time-series or spatio-temporal dataset,  we seek to identify an accurate governing system which respects the intrinsic differential structure. The unknown governing model is parameterized by using both (shallow) multilayer perceptrons and nonlinear differential terms, in order to incorporate relevant correlations between spatio-temporal samples. We demonstrate the approach on several examples where the data is sampled from various dynamical systems and give a comparison to recurrent networks and other data-discovery methods. In addition, we show that for SVHN, MNIST, Fashion MNIST, and CIFAR10/100, our approach lowers the parameter cost as compared to other deep neural networks.

APA


Sun, Y., Zhang, L. & Schaeffer, H.. (2020). NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data. Proceedings of The First Mathematical and Scientific Machine Learning Conference, in Proceedings of Machine Learning Research 107:352-372 Available from https://proceedings.mlr.press/v107/sun20a.html.

Related Material

Download PDF