article

Nonlinear least squares and Sobolev gradients

Author:

Robert J. RenkaAuthors Info & Claims

Applied Numerical Mathematics, Volume 65

Pages 91 - 104

https://doi.org/10.1016/j.apnum.2012.12.002

Published: 01 March 2013 Publication History

Abstract

Least squares methods are effective for solving systems of partial differential equations. In the case of nonlinear systems the equations are usually linearized by a Newton iteration or successive substitution method, and then treated as a linear least squares problem. We show that it is often advantageous to form a sum of squared residuals first, and then compute a zero of the gradient with a Newton-like method. We present an effective method, based on Sobolev gradients, for treating the nonlinear least squares problem directly. The method is based on trust-region subproblems defined by a Sobolev norm and solved by a preconditioned conjugate gradient method with an effective preconditioner that arises naturally from the Sobolev space setting. The trust-region method is shown to be equivalent to a Levenberg-Marquardt method which blends a Newton or Gauss-Newton iteration with a gradient descent iteration, but uses a Sobolev gradient in place of the Euclidean gradient. We also provide an introduction to the Sobolev gradient method and discuss its relationship to operator preconditioning with equivalent operators.

References

[1]

Babuška, I., Error bounds for finite element methods. Numer. Math. v16. 322-333.

Digital Library

[2]

C. Beasley, Finite element solution to nonlinear partial differential equations, Ph.D. thesis, University of North Texas, Denton, TX, 1981.

[3]

Bochev, P.B. and Gunzburger, M.D., Least-Squares Finite Element Methods. 2009. Applied Mathematical Sciences, 2009.Springer.

[4]

Cai, Z., Lazarov, R., Manteuffel, T. and McCormick, S., First-order system least squares for second-order partial differential equations. SIAM J. Numer. Anal. v31. 1785-1799.

Digital Library

[5]

Cai, Z., Manteuffel, T. and McCormick, S., First-order system least squares for second-order partial differential equations. SIAM J. Numer. Anal. v34. 425-454.

Digital Library

[6]

Calder, J., Mansouri, A. and Yezzi, A., Image sharpening via Sobolev gradient flows. SIAM J. Imaging Sci. v3. 981-1014.

Digital Library

[7]

Carey, G.F. and Jiang, B.N., Element-by-element linear and nonlinear solution schemes. Commun. Appl. Numer. Methods. v2. 145-153.

[8]

Carey, G.F. and Jiang, B.N., Least-squares finite element method and preconditioned conjugate gradient solution. Int. J. Numer. Methods Eng. v24. 1283-1296.

[9]

Carey, G.F. and Jiang, B.N., Nonlinear preconditioned conjugate gradient and least-squares finite elements. Comput. Methods Appl. Mech. Eng. v62. 145-154.

Digital Library

[10]

Carey, G.F. and Richardson Jr., W.B., A note on least squares methods. Commun. Numer. Methods Eng. v22. 83-92.

[11]

R. Clough, J. Tocher, Finite element stiffness matrices for analysis of plates in bending, in: Proc. of Conference on Matrix Methods in Structural Analysis, Wright-Patterson AFB, 1965.

[12]

Danaila, I. and Kazemi, P., A new Sobolev gradient method for direct minimization of the Gross-Pitaevskii energy with rotation. SIAM J. Sci. Comput. v32. 2447-2467.

Digital Library

[13]

Faber, V., Manteuffel, T.A. and Parter, S.V., On the theory of equivalent operators and application to the numerical solution of uniformly elliptic partial differential operators. Adv. Appl. Math. v11. 109-163.

Digital Library

[14]

Faragó, I. and Karátson, J., Numerical Solution of Nonlinear Elliptic Problems via Preconditioning Operators: Theory and Applications. 2002. Advances in Computation, 2002.NOVA Science Publishers, Inc., New York.

[15]

Goldstein, C.I., Manteuffel, T.A. and Parter, S.V., Preconditioning and boundary conditions without H2 estimates: L2 condition numbers and the distribution of singular values. SIAM J. Numer. Anal. v30. 343-376.

Digital Library

[16]

Jiang, B., The Least-Squares Finite Element Method: Theory and Applications in Computational Fluid Dynamics and Electromagnetics. 1998. Springer-Verlag, Berlin.

[17]

Joubert, W.D., Manteuffel, T.A., Parter, S.V. and Wong, S.P., Preconditioning second-order elliptic operators: experiment and theory. SIAM J. Sci. Stat. Comput. v13. 259-288.

Digital Library

[18]

Karátson, J. and Faragó, I., Variable preconditioning via quasi-Newton methods for nonlinear problems in Hilbert space. SIAM J. Numer. Anal. v41. 1242-1262.

Digital Library

[19]

Karátson, J. and Faragó, I., Preconditioning operators and Sobolev gradients for nonlinear elliptic problems. Comput. Math. Appl. v50. 1077-1092.

Digital Library

[20]

Karátson, J. and Neuberger, J.W., Newton's method in the context of gradients. Electron. J. Differential Equations. v2007. 1-13.

[21]

Kazemi, P., Using projections in a direct minimization scheme for a wide class of energy functionals. Comm. Appl. Nonlinear Anal. v16. 69

[22]

Kazemi, P. and Renka, R.J., A Levenberg-Marquardt method based on Sobolev gradients. Nonlinear Anal.

[23]

P. Kazemi, R.J. Renka, Minimization of the Ginzburg-Landau energy functional by a Sobolev gradient trust-region method, submitted for publication.

[24]

Kufner, A., Weighted Sobolev Spaces. 1985. John Wiley & Sons, New York, NY.

[25]

Mahavier, W.T., A numerical method utilizing weighted Sobolev descent to solve singular differential equations. Nonlinear World. v4. 435-455.

[26]

Manteuffel, T.A. and Parter, S.V., Preconditioning and boundary conditions. SIAM J. Numer. Anal. v27. 656-694.

Digital Library

[27]

Manteuffel, T.A. and Otto, J.S., Optimal equivalent preconditionings. SIAM J. Numer. Anal. v30. 790-812.

Digital Library

[28]

Neuberger, J.W., An iterative method for approximating solutions to nonlinear partial differential equations. In: Lakshmikantham, V. (Ed.), Applied Nonlinear Analysis, Academic Press, New York. pp. 287-297.

[29]

Neuberger, J.W., Sobolev Gradients and Differential Equations. 2010. Springer Lecture Notes in Mathematics, 2010.second ed. Springer-Verlag.

[30]

Nocedal, J. and Wright, S.J., Numerical Optimization. 1999. Springer Series in Operations Research, 1999.Springer-Verlag, New York.

[31]

Raydan, M., The Barzilai and Borwein gradient descent method for the large scale unconstrained minimization problem. SIAM J. Optim. v7. 26-33.

Digital Library

[32]

Raza, N., Sial, S. and Siddiqi, S., Approximating time evolution related to Ginzburg-Landau functionals via Sobolev gradient methods in a finite-element setting. J. Comput. Phys. v229. 1621-1625.

Digital Library

[33]

R.J. Renka, A Sobolev gradient method for treating the steady-state incompressible Navier-Stokes equations, Cent. Eur. J. Math., in press.

[34]

Renka, R.J. and Neuberger, J.W., Minimal surfaces and Sobolev gradients. SIAM J. Sci. Comput. v16. 1412-1427.

Digital Library

[35]

Renka, R.J. and Neuberger, J.W., Sobolev gradients and the Ginzburg-Landau equations. SIAM J. Sci. Comput. v20. 582-590.

[36]

Richardson Jr., W.B., High-order Sobolev preconditioning. Nonlinear Anal. v63. 1779-1787.

[37]

Richardson Jr., W.B., Sobolev preconditioning for image-processing PDEs. Commun. Numer. Methods Eng. v24. 493-504.

[38]

Richardson, W.B., Pardhanani, A.L., Carey, G.F. and Ardelea, A., Numerical effects in the simulation of Ginzburg-Landau models for superconductivity. Int. J. Numer. Methods Eng. v59. 1251-1272.

[39]

Sial, S., A Sobolev gradient algorithm for minimum energy states of s-wave superconductors-finite element setting. Supercond. Sci. Technol. v18. 675

[40]

S. Sial, J.W. Neuberger, T. Lookman, A. Saxena, Energy minimization using Sobolev gradients-finite-element setting, in: A.D.R. Choudary, F.M. Bhatti (Eds.), Proc. World Conference on 21st Century Mathematics, Lahore, Pakistan, 2005.

[41]

Steihaug, T., The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. v20. 626-637.

[42]

Troesch, B.A., A simple approach to a sensitive two-point boundary value problem. J. Comput. Phys. v21. 279-290.

Cited By

Renka R(2014)A trust region method for constructing triangle-mesh approximations of parametric minimal surfacesApplied Numerical Mathematics10.1016/j.apnum.2013.10.00676(93-100)Online publication date: 1-Feb-2014
https://dl.acm.org/doi/10.1016/j.apnum.2013.10.006
Balima OFavennec YRousse D(2013)Optical tomography reconstruction algorithm with the finite element methodJournal of Computational Physics10.1016/j.jcp.2013.04.043251(461-479)Online publication date: 1-Oct-2013
https://dl.acm.org/doi/10.1016/j.jcp.2013.04.043

Nonlinear least squares and Sobolev gradients

Recommendations

Minimization of the Ginzburg-Landau energy functional by a Sobolev gradient trust-region method

We describe a generalized Levenberg-Marquardt method for computing critical points of the Ginzburg-Landau energy functional which models superconductivity. The algorithm is a blend of a Newton iteration with a Sobolev gradient descent method, and is ...
Convergence theory for the structured BFGS secant method with an application to nonlinear least squares

In 1981, Dennis and Walker developed a convergence theory for structured secant methods which included the PSB and the DFP secant methods but not the straightforward structured version of the BFGS secant method. Here, we fill this gap in the theory by ...
On the local convergence of a derivative-free algorithm for least-squares minimization

In Zhang et al. (accepted by SIAM J. Optim., 2010), we developed a class of derivative-free algorithms, called DFLS, for least-squares minimization. Global convergence of the algorithm as well as its excellent numerical performance within a limited ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Applied Numerical Mathematics

Applied Numerical Mathematics Volume 65, Issue

March, 2013

119 pages

ISSN:0168-9274

Issue’s Table of Contents

Copyright © IMACS © 2012.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 March 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Renka R(2014)A trust region method for constructing triangle-mesh approximations of parametric minimal surfacesApplied Numerical Mathematics10.1016/j.apnum.2013.10.00676(93-100)Online publication date: 1-Feb-2014
https://dl.acm.org/doi/10.1016/j.apnum.2013.10.006
Balima OFavennec YRousse D(2013)Optical tomography reconstruction algorithm with the finite element methodJournal of Computational Physics10.1016/j.jcp.2013.04.043251(461-479)Online publication date: 1-Oct-2013
https://dl.acm.org/doi/10.1016/j.jcp.2013.04.043

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents