Optimal Variance Control of the Score-Function Gradient Estimator for Importance Weighted Bounds

Research output: Contribution to journalConference articleResearchpeer-review

This paper introduces novel results for the score function gradient estimator of the importance weighted variational bound (IWAE). We prove that in the limit of large K (number of importance samples) one can choose the control variate such that the Signal-to-Noise ratio (SNR) of the estimator grows as pK. This is in contrast to the standard pathwise gradient estimator where the SNR decreases as 1/pK. Based on our theoretical findings we develop a novel control variate that extends on VIMCO. Empirically, for the training of both continuous and discrete generative models, the proposed method yields superior variance reduction, resulting in an SNR for IWAE that increases with K without relying on the reparameterization trick. The novel estimator is competitive with state-of-the-art reparameterization-free gradient estimators such as Reweighted Wake-Sleep (RWS) and the thermodynamic variational objective (TVO) when training generative models.

Original languageEnglish
JournalAdvances in Neural Information Processing Systems
Volume2020-December
Number of pages12
ISSN1049-5258
Publication statusPublished - 2020
Event34th Conference on Neural Information Processing Systems, NeurIPS 2020 - Virtual, Online
Duration: 6 Dec 202012 Dec 2020

Conference

Conference34th Conference on Neural Information Processing Systems, NeurIPS 2020
CityVirtual, Online
Period06/12/202012/12/2020
SponsorApple, et al., Microsoft, PDT Partners, Sony Corporation, Tenstorrent

Bibliographical note

Publisher Copyright:
© 2020 Neural information processing systems foundation. All rights reserved.

ID: 276208639