Mask-FPAN: Semi-supervised face parsing in the wild with de-occlusion and UV GAN

Research output: Contribution to journalJournal articleResearchpeer-review


  • Mask-FPAN

    Submitted manuscript, 13.3 MB, PDF document

The field of fine-grained semantic segmentation for a person's face and head, which includes identifying facial parts and head components, has made significant progress in recent years. However, this task remains challenging due to the difficulty of considering ambiguous occlusions and large pose variations. To address these difficulties, we propose a new framework called Mask-FPAN. Our framework includes a de-occlusion module that learns to parse occluded faces in a semi-supervised manner, taking into account face landmark localization, face occlusion estimations, and detected head poses. Additionally, we improve the robustness of 2D face parsing by combining a 3D morphable face model with the UV GAN. We also introduce two new datasets, named FaceOccMask-HQ and CelebAMaskOcc-HQ, to aid in face parsing work. Our proposed Mask-FPAN framework successfully addresses the challenge of face parsing in the wild and achieves significant performance improvements, with a mIoU increase from 0.7353 to 0.9013 compared to the current state-of-the-art on challenging face datasets.

Original languageEnglish
JournalComputers and Graphics (Pergamon)
Pages (from-to)185-193
Number of pages9
Publication statusPublished - 2023

Bibliographical note

Publisher Copyright:
© 2023 Elsevier Ltd

    Research areas

  • 3D face, Face analysis, Face landmark, Face parsing, Generative adversarial network

ID: 366981605