Page 365. Momentum and Optimal Stochastic Search Genevieve B. Orr, Todd K. Leen Department of Com... more Page 365. Momentum and Optimal Stochastic Search Genevieve B. Orr, Todd K. Leen Department of Computer Science and Engineering Oregon Graduate Institute of Science &. Technology 20000 Northwest Walker Road PO Box 91000 Portland. OR 97291-1000 orr@ cse. ogi. ...
Abstract : This reports cover research activity under the grant in place from April 1993 through ... more Abstract : This reports cover research activity under the grant in place from April 1993 through March 1994. Our research addressed algorithms and theory for stochastic learning, non-linear extensions of principal component analysis (PCA) for dimension-reduction, network pruning and methods to incorporate desired invariances into learning. (AN)
There is evidence that loss of memory contributes to poor medication adherence in the elderly [1,... more There is evidence that loss of memory contributes to poor medication adherence in the elderly [1,2]. We previously investigated this contribution in a group of independently-living seniors [3]. Here we describe the application of statistical pattern recognition techniques to medication adherence data, and demonstrate that patterns of adherence can reliably detect mild cognitive loss. We apply neural network classifiers to the task of discriminating between healthy individuals and those with early cognitive loss on the basis of medication adherence behavior. The results establish that data from relatively unobtrusive behavior monitoring can provide reliable inference for individuals.
Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely ... more Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely on a truncated expansion to approximate the time-evolution operator appearing in the master equation. In this note, we give an exact expression for the time-evolution operator for Manhattan learning, a variant of stochastic gradientdescent learning in which the weights are updated in proportion to the sign of the cost function gradient. This closed-form for the time-evolution captures the full non-linearity of the problem without approximation, allowing exact study of the ensemble dynamics. Ensemble Dynamics of Stochastic Learning Stochastic learning algorithms provide recursively refined estimates of optimal model parameters in machine learning, neural networks, adaptive signal processing, and control. Most algorithms are of the form w(n + 1) = w(n) + (n) H(w(n); x(n)) (1) where w(n) 2 R N (with components denoted w i ) is the parameter estimate at the n th iteration of the re...
We present an approach for fusion of video streams produced by multiple imaging sensors such as v... more We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posteriori estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused...
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 2012
Errors in clinical laboratory tests lead to increased costs and patient risks. Such errors are re... more Errors in clinical laboratory tests lead to increased costs and patient risks. Such errors are relatively rare, affecting ∼0.5% of samples. Existing techniques for detecting errors have either far too low sensitivity or specificity to be useful. This preliminary study develops statistical sample selection criteria that capture faults upwards of fifty times more efficiently than expected from random sampling. Although this is only the first step towards an integrated discriminant system for reliable detection of laboratory errors, the statistical detection scheme demonstrated here outperforms existing methods.
The authors extend the theory of search dynamics for stochastic learning algorithms, address the ... more The authors extend the theory of search dynamics for stochastic learning algorithms, address the time evolution of the weight-space probability density and the distribution of convergence times, with particular attention given to escape from local optima, and develop a theoretical framework that describes the evolution of the weight-space probability density. The primary results are exact predictions of the statistical distribution
... MORPHOGENESIS OF THE LATERAL GENICULATE NUCLEUS: HOW SINGULARITIES AFFECT GLOBAL STRUCTURE 13... more ... MORPHOGENESIS OF THE LATERAL GENICULATE NUCLEUS: HOW SINGULARITIES AFFECT GLOBAL STRUCTURE 133 Svilen Tzonev, Klaus Schulten ... OTHER STATISTICAL MODELS FOR BREAST CANCER SURVIVAL 1063 Harry B. Burke, David B. Rosen, Philip H ...
In order to address the highly nonlinear dynamics in estuary flow, we propose a novel data assimi... more In order to address the highly nonlinear dynamics in estuary flow, we propose a novel data assimilation system based on components designed to accurately reflect nonlinear dy- namics. The core of the system is a sigma-point Kalman filter c oupled to a fast, neural network emulator for the flow dynamics. In order to be computa tionally feasible, the entire system operates on a low-dimensional subspace obtained by principal component projec- tion. Our probabilistic latent state space analysis proper ly accounts for noise induced by the dimensionality reduction and by errors in the emulator for the flow dynamics. Experiments on a benchmark estuary problem show that our data assimilation method can significantly reduce prediction errors.
Although the outputs of neural network classifiers are often considered to be estimates of poster... more Although the outputs of neural network classifiers are often considered to be estimates of posterior class probabilities, the literature that assesses the calibration accuracy of these estimates illustrates that practical networks often fall far short of being ideal estimators. The theorems used to justify treating network outputs as good posterior estimates are based on several assumptions: that the network is sufficiently complex to model the posterior distribution accurately, that there are sufficient training data to specify the network, and that the optimization routine is capable of finding the global minimum of the cost function. Any or all of these assumptions may be violated in practice. This article does three things. First, we apply a simple, previously used histogram technique to assess graphically the accuracy of posterior estimates with respect to individual classes. Second, we introduce a simple and fast remapping procedure that transforms network outputs to provide b...
Page 365. Momentum and Optimal Stochastic Search Genevieve B. Orr, Todd K. Leen Department of Com... more Page 365. Momentum and Optimal Stochastic Search Genevieve B. Orr, Todd K. Leen Department of Computer Science and Engineering Oregon Graduate Institute of Science &. Technology 20000 Northwest Walker Road PO Box 91000 Portland. OR 97291-1000 orr@ cse. ogi. ...
Abstract : This reports cover research activity under the grant in place from April 1993 through ... more Abstract : This reports cover research activity under the grant in place from April 1993 through March 1994. Our research addressed algorithms and theory for stochastic learning, non-linear extensions of principal component analysis (PCA) for dimension-reduction, network pruning and methods to incorporate desired invariances into learning. (AN)
There is evidence that loss of memory contributes to poor medication adherence in the elderly [1,... more There is evidence that loss of memory contributes to poor medication adherence in the elderly [1,2]. We previously investigated this contribution in a group of independently-living seniors [3]. Here we describe the application of statistical pattern recognition techniques to medication adherence data, and demonstrate that patterns of adherence can reliably detect mild cognitive loss. We apply neural network classifiers to the task of discriminating between healthy individuals and those with early cognitive loss on the basis of medication adherence behavior. The results establish that data from relatively unobtrusive behavior monitoring can provide reliable inference for individuals.
Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely ... more Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely on a truncated expansion to approximate the time-evolution operator appearing in the master equation. In this note, we give an exact expression for the time-evolution operator for Manhattan learning, a variant of stochastic gradientdescent learning in which the weights are updated in proportion to the sign of the cost function gradient. This closed-form for the time-evolution captures the full non-linearity of the problem without approximation, allowing exact study of the ensemble dynamics. Ensemble Dynamics of Stochastic Learning Stochastic learning algorithms provide recursively refined estimates of optimal model parameters in machine learning, neural networks, adaptive signal processing, and control. Most algorithms are of the form w(n + 1) = w(n) + (n) H(w(n); x(n)) (1) where w(n) 2 R N (with components denoted w i ) is the parameter estimate at the n th iteration of the re...
We present an approach for fusion of video streams produced by multiple imaging sensors such as v... more We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posteriori estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused...
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 2012
Errors in clinical laboratory tests lead to increased costs and patient risks. Such errors are re... more Errors in clinical laboratory tests lead to increased costs and patient risks. Such errors are relatively rare, affecting ∼0.5% of samples. Existing techniques for detecting errors have either far too low sensitivity or specificity to be useful. This preliminary study develops statistical sample selection criteria that capture faults upwards of fifty times more efficiently than expected from random sampling. Although this is only the first step towards an integrated discriminant system for reliable detection of laboratory errors, the statistical detection scheme demonstrated here outperforms existing methods.
The authors extend the theory of search dynamics for stochastic learning algorithms, address the ... more The authors extend the theory of search dynamics for stochastic learning algorithms, address the time evolution of the weight-space probability density and the distribution of convergence times, with particular attention given to escape from local optima, and develop a theoretical framework that describes the evolution of the weight-space probability density. The primary results are exact predictions of the statistical distribution
... MORPHOGENESIS OF THE LATERAL GENICULATE NUCLEUS: HOW SINGULARITIES AFFECT GLOBAL STRUCTURE 13... more ... MORPHOGENESIS OF THE LATERAL GENICULATE NUCLEUS: HOW SINGULARITIES AFFECT GLOBAL STRUCTURE 133 Svilen Tzonev, Klaus Schulten ... OTHER STATISTICAL MODELS FOR BREAST CANCER SURVIVAL 1063 Harry B. Burke, David B. Rosen, Philip H ...
In order to address the highly nonlinear dynamics in estuary flow, we propose a novel data assimi... more In order to address the highly nonlinear dynamics in estuary flow, we propose a novel data assimilation system based on components designed to accurately reflect nonlinear dy- namics. The core of the system is a sigma-point Kalman filter c oupled to a fast, neural network emulator for the flow dynamics. In order to be computa tionally feasible, the entire system operates on a low-dimensional subspace obtained by principal component projec- tion. Our probabilistic latent state space analysis proper ly accounts for noise induced by the dimensionality reduction and by errors in the emulator for the flow dynamics. Experiments on a benchmark estuary problem show that our data assimilation method can significantly reduce prediction errors.
Although the outputs of neural network classifiers are often considered to be estimates of poster... more Although the outputs of neural network classifiers are often considered to be estimates of posterior class probabilities, the literature that assesses the calibration accuracy of these estimates illustrates that practical networks often fall far short of being ideal estimators. The theorems used to justify treating network outputs as good posterior estimates are based on several assumptions: that the network is sufficiently complex to model the posterior distribution accurately, that there are sufficient training data to specify the network, and that the optimization routine is capable of finding the global minimum of the cost function. Any or all of these assumptions may be violated in practice. This article does three things. First, we apply a simple, previously used histogram technique to assess graphically the accuracy of posterior estimates with respect to individual classes. Second, we introduce a simple and fast remapping procedure that transforms network outputs to provide b...
Uploads
Papers by Todd Leen