CPSeg: Finer-grained Image Semantic Segmentation via Chain-of-Thought Language Prompting

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


  • CPSeg

    Accepted author manuscript, 1.67 MB, PDF document

Natural scene analysis and remote sensing imagery offer immense potential for advancements in large-scale language-guided context-aware data utilization. This potential is particularly significant for enhancing performance in downstream tasks such as object detection and segmentation with designed language prompting. In light of this, we introduce the CPSeg (Chain-of-Thought Language Prompting for Finer-grained Semantic Segmentation), an innovative framework designed to augment image segmentation performance by integrating a novel "Chain-of-Thought" process that harnesses textual information associated with images. This groundbreaking approach has been applied to a flood disaster scenario. CPSeg encodes prompt texts derived from various sentences to formulate a coherent chain-of-thought. We use a new vision-language dataset, FloodPrompt, which includes images, semantic masks, and corresponding text information. This not only strengthens the semantic understanding of the scenario but also aids in the key task of semantic segmentation through an interplay of pixel and text matching maps. Our qualitative and quantitative analyses validate the effectiveness of CPSeg.
Original languageEnglish
Title of host publication2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Publication date2024
Publication statusPublished - 2024
EventWACV 2024 - IEEE/CVF Winter Conference on Applications of Computer Vision - Waikola, Hawaii, United States
Duration: 4 Jan 20248 Jan 2024


ConferenceWACV 2024 - IEEE/CVF Winter Conference on Applications of Computer Vision
LandUnited States
ByWaikola, Hawaii

Number of downloads are based on statistics from Google Scholar and www.ku.dk

No data available

ID: 378943255