Jun 28, 2017 · Abstract:We develop a class of algorithms, as variants of the stochastically controlled stochastic gradient (SCSG) methods (Lei and Jordan, ...

We have presented the SCSG method for smooth, non-convex, finite-sum optimization problems. SCSG is the first algorithm that achieves a uniformly better rate ...

The authors propose a stochastically controlled stochastic gradient method for smooth non-convex optimization. They prove a convergence rate for such method ...

•SCSG is the first algorithm that is provably better than SGD;. •SCSG is never worse than any stochastic gradient method in all regimes;.

A class of algorithms, as variants of the stochastically controlled stochastic gradient methods (SCSG) methods, for the smooth non-convex finite-sum ...

We develop a class of algorithms, as variants of the stochastically controlled stochastic gradient (SCSG) methods [21], for the smooth non-convex finite-sum ...

Jun 28, 2017 · We develop and analyze a procedure for gradient-based optimization that we refer to as stochastically controlled stochastic gradient (SCSG). As ...

SCSG. Code for replicating the experiments in the paper Non-Convex Finite-Sum Optimization Via SCSG Methods by Lihua Lei, Cheng Ju, Jianbo Chen, Michael I.

Optimization. Publications. Lihua Lei, Cheng Ju, Jianbo Chen, Michael I. Jordan. Non-Convex Finite-Sum Optimization Via SCSG Methods. In NIPS, 2017. PDF

3 SCSG in Non-convex Optimization. 4 SCSG in Convex Optimization. 2 / 52. Page ... SVRG (in convex finite-sum optimization):. • Also different settings for ...