TY - GEN

T1 - FewGAN

T2 - 29th IEEE International Conference on Image Processing, ICIP 2022

AU - Ben-Moshe, Lior

AU - Benaim, Sagie

AU - Wolf, Lior

N1 - Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - We introduce FewGAN, a generative model for generating novel, high-quality and diverse images whose patch distribution lies in the joint patch distribution of a small number of N > 1 training samples. The method is, in essence, a hierarchical patch-GAN that applies quantization at the first coarse scale, in a similar fashion to VQ-GAN, followed by a pyramid of residual fully convolutional GANs at finer scales. Our key idea is to first use quantization to learn a fixed set of patch embeddings for training images. We then use a separate set of side images to model the structure of generated images using an autoregressive model trained on the learned patch embeddings of training images. Using quantization at the coarsest scale allows the model to generate both conditional and unconditional novel images. Subsequently, a patch-GAN renders the fine details, resulting in high-quality images. In an extensive set of experiments, it is shown that FewGAN outperforms baselines both quantitatively and qualitatively.

AB - We introduce FewGAN, a generative model for generating novel, high-quality and diverse images whose patch distribution lies in the joint patch distribution of a small number of N > 1 training samples. The method is, in essence, a hierarchical patch-GAN that applies quantization at the first coarse scale, in a similar fashion to VQ-GAN, followed by a pyramid of residual fully convolutional GANs at finer scales. Our key idea is to first use quantization to learn a fixed set of patch embeddings for training images. We then use a separate set of side images to model the structure of generated images using an autoregressive model trained on the learned patch embeddings of training images. Using quantization at the coarsest scale allows the model to generate both conditional and unconditional novel images. Subsequently, a patch-GAN renders the fine details, resulting in high-quality images. In an extensive set of experiments, it is shown that FewGAN outperforms baselines both quantitatively and qualitatively.

KW - Few-Shot learning

KW - GANs

KW - Quantization

U2 - 10.1109/ICIP46576.2022.9897704

DO - 10.1109/ICIP46576.2022.9897704

M3 - Article in proceedings

AN - SCOPUS:85146676861

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 751

EP - 755

BT - 2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings

PB - IEEE

Y2 - 16 October 2022 through 19 October 2022

ER -