Efficient block-coordinate descent algorithms for the group lasso

Z Qin, K Scheinberg, D Goldfarb - Mathematical Programming …, 2013 - Springer
Mathematical Programming Computation, 2013Springer
We present two algorithms to solve the Group Lasso problem (Yuan and Lin in, JR Stat Soc
Ser B (Stat Methodol) 68 (1): 49–67, 2006). First, we propose a general version of the Block
Coordinate Descent (BCD) algorithm for the Group Lasso that employs an efficient approach
for optimizing each subproblem exactly. We show that it exhibits excellent performance
when the groups are of moderate size. For groups of large size, we propose an extension of
ISTA/FISTA SIAM (Beck and Teboulle in, SIAM J Imag Sci 2 (1): 183–202, 2009) based on …
Abstract
We present two algorithms to solve the Group Lasso problem (Yuan and Lin in, J R Stat Soc Ser B (Stat Methodol) 68(1):49–67, 2006). First, we propose a general version of the Block Coordinate Descent (BCD) algorithm for the Group Lasso that employs an efficient approach for optimizing each subproblem exactly. We show that it exhibits excellent performance when the groups are of moderate size. For groups of large size, we propose an extension of ISTA/FISTA SIAM (Beck and Teboulle in, SIAM J Imag Sci 2(1):183–202, 2009) based on variable step-lengths that can be viewed as a simplified version of BCD. By combining the two approaches we obtain an implementation that is very competitive and often outperforms other state-of-the-art approaches for this problem. We show how these methods fit into the globally convergent general block coordinate gradient descent framework in Tseng and Yun (Math Program 117(1):387–423, 2009). We also show that the proposed approach is more efficient in practice than the one implemented in Tseng and Yun (Math Program 117(1):387–423, 2009). In addition, we apply our algorithms to the Multiple Measurement Vector (MMV) recovery problem, which can be viewed as a special case of the Group Lasso problem, and compare their performance to other methods in this particular instance.
Springer