Dec 5, 2015 · In this paper, we propose a variance reduction method, called VR-lite, that does not require full gradient computations or extra storage. We ...
Apr 7, 2017 · Abstract. Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, ...
This variance reduction has powerful consequences: it helps VR stochastic methods attain linear convergence rates, and thereby circumvents slowdowns that ...
Variance Reduction for Distributed Stochastic Gradient Descent
ui.adsabs.harvard.edu › abs › abstract
Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, constant stepsizes and preserving ...
Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger stepsizes and preserving linear ...
Mar 15, 2018 · In this paper, we analyze the asynchronous communication protocol in PetuumSGD, and propose a distributed version of variance reduced SGD named DisSVRG.
May 10, 2022 · This article proposes a distributed stochastic algorithm with variance reduction for general smooth non-convex finite-sum optimization, ...
Missing: Descent. | Show results with:Descent.
For non-smooth problems, convergence rate is same as subgradient method. For smooth problems, number of iterations is much higher than gradient descent. Effect ...
In this paper, we develop a general approach of using control variate for variance reduction in stochastic gradient.
Missing: Distributed | Show results with:Distributed
These variance-reduced SGD methods differ from the sampling strategies discussed before in a significant way: they can iteratively learn the stochastic ...