Search Results

As a popular channel pruning method for convolutional neural networks (CNNs), network slimming (NS) has a three-stage process: (1) it trains a CNN with ${\ell }_1$ regularization applied to the scaling factors of the batch normalization layers; (2) it ...

Federated learning is a new distributed machine learning framework, where numerous heterogeneous clients collaboratively train a model without sharing training data. In this work, we consider a practical and ubiquitous issue when deploying federated ...

This paper presents a new convergent Plug-and-Play (PnP) algorithm. PnP methods are efficient iterative algorithms for solving image inverse problems formulated as the minimization of the sum of a data-fidelity term and a regularization term. PnP ...$$

Because of its linear complexity, naive Bayes classification remains an attractive supervised learning method, especially in very large-scale settings. We propose a sparse version of naive Bayes, which can be used for feature selection. This leads to a ...

Solving Rank Constrained Least Squares via Recursive Importance Sketching

In statistics and machine learning, we sometimes run into the rank-constrained least squares problems, for which we need to find the best low-rank fit between sets of data, such as ...

In this paper, we propose a recursive importance sketching algorithm for rank-constrained least squares optimization (RISRO). The key step of RISRO is recursive importance sketching, a new sketching framework based on deterministically designed recursive ...

Understanding the iterative behavior of stochastic optimization algorithms for minimizing nonconvex functions remains a crucial challenge in demystifying deep learning. In particular, it is not yet understood why certain simple techniques are remarkably ...

We consider using gradient descent to minimize the nonconvex function f(X) = ϕ(XX^T) over an n × r factor matrix X, in which ϕ is an underlying smooth convex cost function defined over n × n matrices. While only a second-order stationary point X can be ...

This paper studies accelerated gradient methods for nonconvex optimization with Lipschitz continuous gradient and Hessian. We propose two simple accelerated gradient methods, restarted accelerated gradient descent (AGD) and restarted heavy ball (HB) ...

In this work, we study the performance of sub-gradient method (SubGM) on a natural nonconvex and nonsmooth formulation of low-rank matrix recovery with ℓ₁-loss, where the goal is to recover a low-rank matrix from a limited number of measurements, a ...

We describe a line-search algorithm which achieves the best-known worst-case complexity results for problems with a certain "strict saddle" property that has been observed to hold in low-rank matrix optimization problems. Our algorithm is adaptive, in ...

The advantages of cloud computing attract a large number of enterprises to deploy their applications into the cloud, thereby reducing their own operating costs. This paper considers deploying inelastic applications into the cloud and proposes an optimal ...

Deep learning techniques based on the neural network have made significant achievements in various fields of artificial intelligence. However, model training requires large‐scale data sets, these data sets are crowd‐sourced and model parameters ...

This paper studies a structured compound stochastic program (SP) involving multiple expectations coupled by nonconvex and nonsmooth functions. We present a successive convex programming-based sampling algorithm and establish its subsequential convergence. ...

We introduce a generalized version of the concave Kurdyka-Łojasiewicz (KL) property by employing nonsmooth desingularizing functions. We also present the exact modulus of the generalized concave KL property, which provides an answer to the open question ...

Nonconvex Stochastic Optimization

Nonconvex stochastic optimization problems arise in many machine learning problems, including deep learning. The stochastic gradient Hamiltonian Monte Carlo (SGHMC) is a variant of stochastic gradients with a momentum ...

Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is a variant of stochastic gradients with momentum where a controlled and properly scaled Gaussian noise is added to the stochastic gradients to steer the iterates toward a global minimum. Many works ...

In order to explore some concrete metric relationship between the ‖ x ‖ 1 / ‖ x ‖ ∞ and ‖ x ‖ 0, we introduce a wonderful triangle whose sides are composed of ‖ x ‖ 0 , ‖ x ‖ 1 and ‖ x ‖ ∞ for any non-zero vector x ∈ R n by delving ...

Inspired by the recent developments in risk sharing problems for the value at risk (VaR), the expected shortfall (ES), and the range value at risk (RVaR), we study the optimization of risk sharing for general tail risk measures. Explicit formulas of the ...

Fluence map optimization for intensity-modulated radiation therapy planning can be formulated as a large-scale inverse problem with competing objectives and constraints associated with the tumors and organs at risk. Unfortunately, clinically relevant dose–...

This paper investigates a problem of broad practical interest, namely, the reconstruction of a large-dimensional low-rank tensor from highly incomplete and randomly corrupted observations of its entries. Although a number of papers have been ...

We study a noisy tensor completion problem of broad practical interest, namely, the reconstruction of a low-rank tensor from highly incomplete and randomly corrupted observations of its entries. Whereas a variety of prior work has been dedicated to this ...

Given the noisy pairwise measurements among a set of unknown group elements, how does one recover them efficiently and robustly? This problem, known as group synchronization, has drawn tremendous attention in the scientific community. In this work, we ...