Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Adaptive Douglas--Rachford Splitting Algorithm from a Yosida Approximation Standpoint

Published: 01 January 2021 Publication History

Abstract

The adaptive Douglas--Rachford splitting algorithm iteratively applies the operator $T=\kappa_{n}Q_{A}Q_{B}+(1-\kappa_{n}){Id}$ to solve the inclusion problem $\text{zer}(A+B)$. By taking a Yosida approximation standpoint, we express in canonical form $Q_{A}Q_{B}=({Id}-(\gamma+\lambda)\ ^{\gamma}\!A)\circ({Id}-(\gamma+\lambda)\ ^{\lambda}\!B)$. We extend the domain of indices $\gamma, \lambda$ to the entire real line, so that the adaptive algorithm is able to encompass a forward-backward splitting algorithm into one unified framework. Convergence results for both primal and dual problems are proved for different combinations of (strongly and weakly) monotone and comonotone operators. Under the “monotone + comonotone” assumption, we obtain a better rate bound for linear convergence than the classical Douglas--Rachford algorithm.

References

[1]
S. Bartz, R. Campoy, and H. M. Phan, Demiclosedness principles for generalized nonexpansive mappings, J. Optim. Theory Appl., 186 (2020), pp. 759--778, https://doi.org/10.1007/s10957-020-01734-6.
[2]
S. Bartz, M. N. Dao, and H. M. Phan, Conical Averagedness and Convergence Analysis of Fixed Point Algorithms, preprint, https://arxiv.org/abs/1910.14185, 2019.
[3]
H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, 2017.
[4]
C. Botsaris and D. Jacobson, A Newton-type curvilinear search method for optimization, J. Math. Anal. Appl., 54 (1976), pp. 217--229, https://doi.org/10.1016/0022-247X(76)90246-8.
[5]
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learn., 3 (2011), pp. 1--122, https://doi.org/10.1561/2200000016.
[6]
P. L. Combettes and T. Pennanen, Proximal methods for cohypomonotone operators, SIAM J. Control Optim., 43 (2004), pp. 731--742, https://doi.org/10.1137/S0363012903427336.
[7]
M. N. Dao and H. M. Phan, Adaptive Douglas--Rachford splitting algorithm for the sum of two operators, SIAM J. Optim., 29 (2019), pp. 2697--2724, https://doi.org/10.1137/18M121160X.
[8]
J. Douglas and H. H. Rachford, On the numerical solution of heat conduction problems in two and three space variables, Trans. Amer. Math. Soc., 82 (1956), pp. 421--439.
[9]
J. Eckstein, Splitting Methods for Monotone Operators with Applications to Parallel Optimization, Ph.D. thesis, Massachusetts Institute of Technology, 1989.
[10]
D. Gabay, Chapter IX: Applications of the method of multipliers to variational inequalities, in Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, Stud. Math. Appl. 15, Elsevier, 1983, pp. 299--331.
[11]
P. Giselsson, Tight global linear convergence rate bounds for Douglas--Rachford splitting, J. Fixed Point Theory Appl., 19 (2017), pp. 2241--2270.
[12]
A. A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), pp. 709--710, https://projecteuclid.org:443/euclid.bams/1183526263.
[13]
K. Guo, D. Han, and X. Yuan, Convergence analysis of Douglas--Rachford splitting method for “strongly + weakly” convex programming, SIAM J. Numer. Anal., 55 (2017), pp. 1549--1577, https://doi.org/10.1137/16M1078604.
[14]
M. R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303--320.
[15]
P. L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), pp. 964--979, https://doi.org/10.1137/0716071.
[16]
D. W. Peaceman and H. H. Rachford, Jr., The numerical solution of parabolic and elliptic differential equations, J. Soc. Indust. Appl. Math., 3 (1955), pp. 28--41, https://doi.org/10.1137/0103003.
[17]
M. J. D. Powell, A method for nonlinear constraints in minimization problems, in Optimization (Sympos., Univ. Keele, Keele, UK), Academic Press, 1969, pp. 283--298.
[18]
R. T. Rockafellar, Augmented Lagrangians and applications of the proximal point algorithm in convex programming, Math. Oper. Res., 1 (1976), pp. 97--116, http://www.jstor.org/stable/3689277.
[19]
R. T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), pp. 877--898, https://doi.org/10.1137/0314056.
[20]
W. Rudin, Functional Analysis, McGraw-Hill, 1991.
[21]
B. F. Svaiter, On weak convergence of the Douglas--Rachford method, SIAM J. Control Optim., 49 (2011), pp. 280--287, https://doi.org/10.1137/100788100.
[22]
J. von Neumann, Functional Operators II: The Geometry of Orthogonal Spaces, Ann. Math. Stud. 21, Princeton University Press, 1950, http://www.jstor.org/stable/j.ctt1bc543b.

Cited By

View all
  • (2022)An Adaptive Alternating Direction Method of MultipliersJournal of Optimization Theory and Applications10.1007/s10957-022-02098-9195:3(1019-1055)Online publication date: 1-Dec-2022

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization  Volume 31, Issue 3
DOI:10.1137/sjope8.31.3
Issue’s Table of Contents

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 January 2021

Author Tags

  1. adaptive Douglas--Rachford algorithm
  2. Yosida approximation
  3. forward-backward algorithm
  4. inclusion problem
  5. linear convergence
  6. weak monotonicity
  7. comonotone
  8. resolvent

Author Tags

  1. Primary
  2. 47H10
  3. 49M27; Secondary
  4. 41A25
  5. 65K05
  6. 65K10

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 10 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2022)An Adaptive Alternating Direction Method of MultipliersJournal of Optimization Theory and Applications10.1007/s10957-022-02098-9195:3(1019-1055)Online publication date: 1-Dec-2022

View Options

View options

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media