Open access
Author
Date
2021Type
- Doctoral Thesis
ETH Bibliography
yes
Altmetrics
Abstract
Humans understand the world through concepts. They form high-level
abstractions to represent sensory information in a simple way. Conceptual
thinking is one of the central aspects of human intelligence as it allows
knowledge reuse, simplifies the understanding of cause-effect relationships,
and empowers creativity. We argue that further progress in our
quest for artificial intelligence critically depends on the development of
machine learning algorithms that can infer concepts from data and fantasize
new data based on those concepts. Deep generative models with latent
variables (DGLs) provide a unified framework for both (i) representation
learning and (ii) data synthesis. Despite remarkable recent progress in this
area, many practical challenges prevent DGLs from attaining their full potential.
The goal of this thesis is to highlight those challenges and propose
novel algorithmic solutions to address them.
The first part of this thesis studies DGLs in the context of sequential data
such as text sequences. DGLs for sequences are typically trained via maximum
likelihood estimation (MLE), but this renders models with uninformative
latent variables. To regularize degenerate MLE training, we propose
an importance-weighted dropout scheme, implemented using an adversarial
approach. In contrast to standard dropout, our method obtains a better
trade-off between representation learning and sequence modeling. In
the second part, we discuss DGLs for images in the form of variational
autoencoders (VAEs). VAEs are generally regarded as inferior to other generative
models concerning both density estimation and image generation
quality. By augmenting VAEs with the newly introduced spatial dependency
layers, we considerably close this performance gap that hinders the
widespread adoption of VAEs. In the third part, we integrate our image
VAE into a framework for automatic analysis of sleep patterns from brain
signals. We apply VAE to detect anomalies in sleep recordings and combine
it with a supervised classifier of sleep stages. The resulting framework
is implemented as a web server, serving sleep labs around the world. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000530883Publication status
publishedExternal links
Search print copy at ETH Library
Publisher
ETH ZurichSubject
Deep learning; Generative modeling; Representation learningOrganisational unit
03659 - Buhmann, Joachim M. (emeritus) / Buhmann, Joachim M. (emeritus)
More
Show all metadata
ETH Bibliography
yes
Altmetrics