Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Fast Moreau envelope computation I: numerical algorithms

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The present article summarizes the state of the art algorithms to compute the discrete Moreau envelope, and presents a new linear-time algorithm, named NEP for NonExpansive Proximal mapping. Numerical comparisons between the NEP and two existing algorithms: The Linear-time Legendre Transform (LLT) and the Parabolic Envelope (PE) algorithms are performed. Worst-case time complexity, convergence results, and examples are included. The fast Moreau envelope algorithms first factor the Moreau envelope as several one-dimensional transforms and then reduce the brute force quadratic worst-case time complexity to linear time by using either the equivalence with Fast Legendre Transform algorithms, the computation of a lower envelope of parabolas, or, in the convex case, the non expansiveness of the proximal mapping.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Attouch, H., Wets, R.J.-B.: A convergence theory for saddle functions. Trans. Am. Math. Soc. 280, 1–41 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  2. Aurell, E., Gurbatov, S.-g. N., Simdyankin, S.-g.I.: Numerical proof of self-similarity in Burgers’ turbulence. Tech. Rep. http://arxiv.org/abs/patt-sol/9602005,arXiv.org (1996)

  3. Bec, J., Frisch, U., Khanin, K.: Kicked Burger turbulence. J. Fluid Mech. 416, 239–267 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bernard, F., Thibault, L.: Prox-regularity of functions and sets in Banach spaces. Set.-Valued Anal. 12, 25–47 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  5. Bernard, F., Thibault, L.: Prox-regular functions in Hilbert spaces. J. Math. Anal. Appl. 303, 1–14 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  6. Bernard, F., Thibault, L.: Uniform prox-regularity of functions and epigraphs in Hilbert spaces. Nonlinear Anal. 60, 187–207 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  7. Bonnans, J.F., Gilbert, J.C., Lemaréchal, C., Sagastizábal, C.A.: A family of variable metric proximal methods. Math. Program. 68, 15–47 (1995)

    Google Scholar 

  8. Brenier, Y.: Un algorithme rapide pour le calcul de transformées de Legendre–Fenchel discrètes. C. R. Acad. Sci. Paris Sér. I. Math. 308, 587–589 (1989)

    MATH  MathSciNet  Google Scholar 

  9. Burke, J.V., Qian, M.: A variable metric proximal point algorithm for monotone operators. Tech. Rep., Department of Mathematics, University of Washington, Seattle, Washington; and Department of Mathematics, California State University, Fullerton, CA (1992)

  10. Burke, J.V., Qian, M.: On the superlinear convergence of the variable metric proximal point algorithm using Broyden and BFGS matrix secant updating. Tech. Rep., Department of Mathematics, University of Washington, Seattle, Washington; and Department of Mathematics, California State University, Fullerton, CA (1996)

  11. Burke, J.V., Qian, M.: On the local super-linear convergence of a matrix secant implementation of the variable metric proximal point algorithm for monotone operators. Tech. Rep., Department of Mathematics, University of Washington, Seattle, Washington; and Department of Mathematics, California State University, Fullerton, CA (1998)

  12. Clarke, F.H., Stern, R.J., Wolenski, P.R.: Proximal smoothness and the lower-C 2 property. J. Convex Anal. 2, 117–144 (1995)

    MATH  MathSciNet  Google Scholar 

  13. Cominetti, R.: Coupling the proximal point algorithm with approximation methods. J. Optim. Theory Appl. 95, 581–600 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  14. Corrias, L.: Fast Legendre–Fenchel transform and applications to Hamilton–Jacobi equations and conservation laws. SIAM J. Numer. Anal. 33, 1534–1558 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  15. Deniau, L.: Proposition d’un opérateur géométrique pour l’analyse et l’identification de signaux et images. PhD thesis, Université de Paris-Sud, Centre d’Orsay (Dec. 1997)

  16. Deniau, L., Blanc-Talon, J.: Fractal analysis with Hausdorff distance under affine transformations, Tech. Rep., ETCA-CREA-SP (1995)

  17. Eckstein, J.: Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming. Math. Oper. Res. 18, 202–226 (1993)

    MATH  MathSciNet  Google Scholar 

  18. Felzenszwalb, P.F., Huttenlocher, D.P.: Distance transforms of sampled functions, Tech. Rep. TR2004-1963, Cornell Computing and Information Science (Sept. 2004)

  19. Frisch, U., Bec, J.: Burgulence. In: Lesieur, A.M., David, F. (eds.) Les Houches 2000: New Trends in Turbulence, pp. 341–383. Springer EDP-Sciences, France (2001)

    Google Scholar 

  20. Frisch, U., Bec, J., Villone, B.: Singularities and the distribution of density in the Burgers/adhesion model. Physica D. 152–153, 620–635 (2001)

    Article  MathSciNet  Google Scholar 

  21. Güler, O.: On the convergence of the proximal point algorithm for convex minimization. SIAM J. Control Optim. 29, 403–419 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  22. Gurbatov, S.N., Simdyankin, S.I., Aurell, E., Frisch, U., Tóth, G.: On the decay of Burgers turbulence. J. Fluid Mech. 344, 339–374 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  23. Gurbatov, S.N., Troussov, A.V.: The decay of multiscale signals – deterministic model of the Burgers turbulence, Tech. Rep., arXiv.org (2000)

  24. Hamdi, A.: A Moreau-Yosida regularization of a difference of two convex functions. Appl. Math. E.-Notes 5, 164–170 (2005) (electronic)

    MATH  MathSciNet  Google Scholar 

  25. Hare, W.L., Sagastizabal, C.: Computing proximal points on nonconvex functions, Tech. Rep., Optimization Online Digest – June 2005 (2005)

  26. Helluy, P.: Simulation numérique des écoulements multiphasiques : De la théorie aux applications, PhD thesis, Institut des Sciences de l’Ingenieur de Toulon et du Var, Laboratoire Modélisation Numérique et Couplages, BP 56, 83162 La Valette CEDEX, France Habilitation à Diriger des Recherches (Jan. 2005)

  27. Hiriart-Urruty, J.B.: Lipschitz r-continuity of the approximate subdifferential of a convex function. Math. Scand. 47, 123–134 (1980)

    MATH  MathSciNet  Google Scholar 

  28. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex analysis and minimization algorithms. In: Grundlehren der Mathematischen Wissenschaften, vol. 305–306 [Fundamental Principles of Mathematical Sciences] Springer, Berlin Heidelberg New York (1993) (Vol. I: Fundamentals, Vol. II: Advanced theory and bundle methods)

    Google Scholar 

  29. Hisakado, T., Okumura, K., Vukadinovic, V., Trajkovic, L.: Characterization of a simple communication network using Legendre transform. In: Proc. IEEE Int. Symp. Circuits and Systems, vol. 3, pp. 738–741 (May 2003)

  30. Iusem, A.N., Svaiter, B.F., Teboulle, M.: Entropy-like proximal methods in convex programming. Math. Oper. Res. 19, 790–814 (1994)

    MATH  MathSciNet  Google Scholar 

  31. Koopen, B.: Contact of bodies in 2D-space: Implementing the Discrete Legendre Transform. AI Master’s thesis, Intelligent Autonomous Systems Group, University of Amsterdam (Feb. 2002)

  32. Legras, B., Pisso, I., Berthet, G., Lefèvre, F.: Variability of the Lagrangian turbulent diffusion in the lower stratosphere. Atmos. Chem. Phys. 5, 1605–1622 (2005)

    Article  Google Scholar 

  33. Lehdili, N., Moudafi, A.: Combining the proximal algorithm and Tikhonov regularization. Optimization 37, 239–252 (1996)

    MATH  MathSciNet  Google Scholar 

  34. Lemaire, B.: The proximal algorithm. In: New Methods in Optimization and Their Industrial Uses (Pau/Paris, 1987) Internat, vol. 87, pp. 73–87. Schriftenreihe Numer. Math., Birkhäuser, Basel (1989)

    Google Scholar 

  35. Lemaréchal, C., Oustry, F.: Growth conditions and \(\mathcal{U}\)-Lagrangians. Set.-Valued Anal. 9, 123–129 (2001) (Wellposedness in optimization and related topics (Gargnano, 1999)

    Article  MATH  MathSciNet  Google Scholar 

  36. Lemaréchal, C., Oustry, F., Sagastizábal, C.: The \(\mathcal{U}\)-Lagrangian of a convex function. Trans. Am. Math. Soc. 352, 711–729 (2000)

    Article  MATH  Google Scholar 

  37. Lemaréchal, C., Sagastizábal, C.: More than first-order developments of convex functions: primal-dual relations. J. Convex Anal. 3, 255–268 (1996)

    MATH  MathSciNet  Google Scholar 

  38. Lemaréchal, C., Sagastizábal, C.: Practical aspects of the Moreau–Yosida regularization: theoretical preliminaries. SIAM J. Optim. 7, 367–385 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  39. Lemaréchal, C., Sagastizábal, C.: Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76, 393–410 (1997)

    Article  Google Scholar 

  40. Lucet, Y.: A fast computational algorithm for the Legendre–Fenchel transform. Comput. Optim. Appl. 6, 27–57 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  41. Lucet, Y.: Faster than the Fast Legendre Transform, the Linear-time Legendre Transform. Numer. Algor. 16, 171–185 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  42. Lucet, Y.: Fast Moreau envelope computation II: Applications. Tech. Rep., University of British Columbia, Okanagan (2005)

    Google Scholar 

  43. Lucet, Y.: A linear Euclidean distance transform algorithm based on the Linear-time Legendre Transform. In: Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV 2005), Victoria BC, May 2005. IEEE Computer Society Press, Los Alamitos, CA

  44. Meng, F., Sun, D., Zhao, G.: Semismoothness of solutions to generalized equations and the Moreau–Yosida regularization. Math. Program. (2005) (online first)

  45. Meng, F., Zhao, G.: On second-order properties of the Moreau–Yosida regularization for constrained nonsmooth convex programs. Numer. Funct. Anal. Optim. 25, 515–529 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  46. Meng, F.W., Hao, Y.: Piecewise smoothness for Moreau–Yosida approximation to a piecewise C 2 convex function. Adv. Math. (China), 30, 354–358 (2001)

    MATH  MathSciNet  Google Scholar 

  47. Mifflin, R.: A quasi-second-order proximal bundle algorithm. Math. Program. 73, 51–72 (1996)

    Article  MathSciNet  Google Scholar 

  48. Mifflin, R., Qi, L., Sun, D.: Properties of the Moreau-Yosida regularization of a piecewise C 2 convex function. Math. Program. 84, 269–281 (1999)

    MATH  MathSciNet  Google Scholar 

  49. Mifflin, R., Sagastizábal, C.: \(\mathcal{VU}\)-decomposition derivatives for convex max-functions. In: Ill-posed Variational Problems and Regularization Techniques (Trier, 1998). Lecture Notes in Economics and Mathematical Systems, vol. 477, pp. 167–186. Springer, Berlin Heidelberg New York (1999)

    Google Scholar 

  50. Mifflin, R., Sagastizábal, C.: Functions with primal-dual gradient structure and \(\mathcal{U}\)-Hessians. In: Nonlinear Optimization and Related Topics (Erice, 1998). Applied Optimization, vol. 36, pp. 219–233. Kluwer, Dordrecht (2000)

    Google Scholar 

  51. Mifflin, R., Sagastizábal, C.: On \(\mathcal{VU}\)-theory for functions with primal-dual gradient structure. SIAM J. Optim. 11, 547–571 (2000) (electronic)

    Article  MATH  MathSciNet  Google Scholar 

  52. Mifflin, R., Sagastizábal, C.: Primal-dual gradient structured functions: second-order results; links to epi-derivatives and partly smooth functions. SIAM J. Optim. 13, 1174–1194 (2003) (electronic)

    Article  MATH  MathSciNet  Google Scholar 

  53. Mifflin, R., Sagastizábal, C.: On the relation between \(\mathcal{U}\)-Hessians and second-order epi-derivatives. Eur. J. Oper. Res. 157, 28–38 (2004)

    Article  MATH  Google Scholar 

  54. Mifflin, R., Sagastizábal, C.: \(\mathcal{VU}\)-smoothness and proximal point results for some nonconvex functions. Optim. Methods Softw. 19, 463–478 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  55. Mifflin, R., Sagastizábal, C.: Relating \(\mathcal{U}\)-Lagrangians to second-order epi-derivatives and proximal-tracks. J. Convex Anal. 12, 81–93 (2005)

    MATH  MathSciNet  Google Scholar 

  56. Mifflin, R., Sagastizábal, C.: A \(\mathcal{VU}\)-algorithm for convex minimization. Math. Program. (2005)

  57. Mifflin, R., Sun, D., Qi, L.: Quasi-Newton bundle-type methods for nondifferentiable convex optimization. SIAM J. Optim. 8, 583–603 (1998) (electronic)

    Article  MATH  MathSciNet  Google Scholar 

  58. Moreau, J.-J.: Propriétés des applications “prox”. C. R. Acad. Sci. Paris 256, 1069–1071 (1963)

    MATH  MathSciNet  Google Scholar 

  59. Moreau, J.-J.: Proximité et dualité dans un espace Hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)

    MATH  MathSciNet  Google Scholar 

  60. Moreau, J.-J.: Convexity and duality. In: Functional Analysis and Optimization, pp. 145–169. Academic, New York (1966)

    Google Scholar 

  61. Moudafi, A.: Coupling proximal methods and variational convergence. Z. Oper. Res. 38, 269–280 (1993)

    MATH  MathSciNet  Google Scholar 

  62. Moudafi, A.: Coupling proximal algorithm and Tikhonov method. Nonlinear Times Digest 1, 203–209 (1994)

    MATH  MathSciNet  Google Scholar 

  63. Noullez, A., Gurbatov, S.N., Aurell, E., Simdyankin, S.I.: The global picture of self-similar and not self-similar decay in Burgers turbulence. Tech. Rep. nlin.CD/0409022, arXiv.org eprint archive, (Sept. 2004)

  64. Noullez, A., Vergassola, M.: A fast Legendre transform algorithm and applications to the adhesion model. J. Sci. Comput. 9, 259–281 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  65. Poliquin, R., Rockafellar, R.T.: Generalized Hessian properties of regularized nonsmooth functions. SIAM J. Optim. 6, 1121–1137 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  66. Poliquin, R., Rockafellar, R.T.: Prox-regular functions in variational analysis. Trans. Am. Math. Soc. 348, 1805–1838 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  67. Qi, L.: Second-order analysis of the Moreau-Yosida regularization. In: Nonlinear analysis and convex analysis (Niigata, 1998), pp. 16–25. World Sci., River Edge, NJ (1999)

    Google Scholar 

  68. Qi, L., Chen, X.: A preconditioning proximal Newton method for nondifferentiable convex optimization. Math. Program. 76, 411–429 (1997)

    Article  MathSciNet  Google Scholar 

  69. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton, NY (1970)

    MATH  Google Scholar 

  70. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  71. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin Heidelberg New York (1998)

    MATH  Google Scholar 

  72. She, Z.-S., Aurell, E., Frisch, U.: The inviscid Burgers equation with initial data of Brownian type. Comm. Math. Phys. 148, 623–641 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  73. Spingarn, J.E.: Submonotone mappings and the proximal point algorithm. Numer. Funct. Anal. Optim. 4, 123–150 (1981–1982)

    MathSciNet  Google Scholar 

  74. Teboulle, M.: Entropic proximal mappings with applications to nonlinear programming. Math. Oper. Res. 17, 670–690 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  75. Vergassola, M., Dubrulle, B., Frisch, U., Noullez, A.: Burger’s equation, devil’s staircases and the mass distribution function for large-scale structures. Astron. Astrophys. 289, 325–356 (1994)

    Google Scholar 

  76. Wei, Z., Qi, L.: Convergence analysis of a proximal Newton method. Numer. Funct. Anal. Optim. 17, 463–472 (1996)

    MATH  MathSciNet  Google Scholar 

  77. Yosida, K.: Functional Analysis, Classics in Mathematics. Springer, Berlin Heidelberg New York (1995) (Reprint of the sixth (1980) edition)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science, I. K. Barber School of Arts and Sciences-Unit 4, The University of British Columbia Okanagan, 3333 University Way, Kelowna, BC V1V 1V7, Canada

    Yves Lucet

Authors
  1. Yves Lucet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yves Lucet.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lucet, Y. Fast Moreau envelope computation I: numerical algorithms. Numer Algor 43, 235–249 (2006). https://doi.org/10.1007/s11075-006-9056-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-006-9056-0

Keywords

Mathematics Subject Classifications (2000)

Search

Navigation