Ladder variational autoencoders

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Standard

Ladder variational autoencoders. / Sønderby, Casper Kaae; Raiko, Tapani; Maaløe, Lars; Sønderby, Søren Kaae; Winther, Ole.

Advances in Neural Information Processing Systems 29 (NIPS 2016). ed. / D. D. Lee; M. Sugiyama; U. V. Luxburg; I. Guyon; R. Garnett. Curran Associates, Inc., 2016. p. 3745-3753 (Advances in Neural Information Processing Systems, Vol. 29).

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Harvard

Sønderby, CK, Raiko, T, Maaløe, L, Sønderby, SK & Winther, O 2016, Ladder variational autoencoders. in DD Lee, M Sugiyama, UV Luxburg, I Guyon & R Garnett (eds), Advances in Neural Information Processing Systems 29 (NIPS 2016). Curran Associates, Inc., Advances in Neural Information Processing Systems, vol. 29, pp. 3745-3753, 30th Annual Conference on Neural Information Processing Systems, Barcelona, Spain, 05/12/2016. <https://papers.nips.cc/paper/6275-ladder-variational-autoencoders>

APA

Sønderby, C. K., Raiko, T., Maaløe, L., Sønderby, S. K., & Winther, O. (2016). Ladder variational autoencoders. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 29 (NIPS 2016) (pp. 3745-3753). Curran Associates, Inc.. Advances in Neural Information Processing Systems Vol. 29 https://papers.nips.cc/paper/6275-ladder-variational-autoencoders

Vancouver

Sønderby CK, Raiko T, Maaløe L, Sønderby SK, Winther O. Ladder variational autoencoders. In Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R, editors, Advances in Neural Information Processing Systems 29 (NIPS 2016). Curran Associates, Inc. 2016. p. 3745-3753. (Advances in Neural Information Processing Systems, Vol. 29).

Author

Sønderby, Casper Kaae ; Raiko, Tapani ; Maaløe, Lars ; Sønderby, Søren Kaae ; Winther, Ole. / Ladder variational autoencoders. Advances in Neural Information Processing Systems 29 (NIPS 2016). editor / D. D. Lee ; M. Sugiyama ; U. V. Luxburg ; I. Guyon ; R. Garnett. Curran Associates, Inc., 2016. pp. 3745-3753 (Advances in Neural Information Processing Systems, Vol. 29).

Bibtex

@inproceedings{f87d558a727f49639399a8f5f6f92567,
title = "Ladder variational autoencoders",
abstract = "Variational autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently proposed Ladder Network. We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other generative models. We provide a detailed analysis of the learned hierarchical latent representation and show that our new inference model is qualitatively different and utilizes a deeper more distributed hierarchy of latent variables. Finally, we observe that batch-normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.",
author = "S{\o}nderby, {Casper Kaae} and Tapani Raiko and Lars Maal{\o}e and S{\o}nderby, {S{\o}ren Kaae} and Ole Winther",
year = "2016",
language = "English",
series = "Advances in Neural Information Processing Systems",
publisher = "Curran Associates, Inc.",
pages = "3745--3753",
editor = "Lee, {D. D.} and M. Sugiyama and Luxburg, {U. V.} and I. Guyon and R. Garnett",
booktitle = "Advances in Neural Information Processing Systems 29 (NIPS 2016)",
note = "null ; Conference date: 05-12-2016 Through 10-12-2016",

}

RIS

TY - GEN

T1 - Ladder variational autoencoders

AU - Sønderby, Casper Kaae

AU - Raiko, Tapani

AU - Maaløe, Lars

AU - Sønderby, Søren Kaae

AU - Winther, Ole

N1 - Conference code: 30

PY - 2016

Y1 - 2016

N2 - Variational autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently proposed Ladder Network. We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other generative models. We provide a detailed analysis of the learned hierarchical latent representation and show that our new inference model is qualitatively different and utilizes a deeper more distributed hierarchy of latent variables. Finally, we observe that batch-normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.

AB - Variational autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently proposed Ladder Network. We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other generative models. We provide a detailed analysis of the learned hierarchical latent representation and show that our new inference model is qualitatively different and utilizes a deeper more distributed hierarchy of latent variables. Finally, we observe that batch-normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.

UR - http://www.scopus.com/inward/record.url?scp=85019264158&partnerID=8YFLogxK

M3 - Article in proceedings

AN - SCOPUS:85019264158

T3 - Advances in Neural Information Processing Systems

SP - 3745

EP - 3753

BT - Advances in Neural Information Processing Systems 29 (NIPS 2016)

A2 - Lee, D. D.

A2 - Sugiyama, M.

A2 - Luxburg, U. V.

A2 - Guyon, I.

A2 - Garnett, R.

PB - Curran Associates, Inc.

Y2 - 5 December 2016 through 10 December 2016

ER -

ID: 179322079