The ignorant led by the blind: A hybrid human-machine vision system for fine-grained categorization
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
The ignorant led by the blind : A hybrid human-machine vision system for fine-grained categorization. / Branson, Steve; Van Horn, Grant; Wah, Catherine; Perona, Pietro; Belongie, Serge.
In: International Journal of Computer Vision, Vol. 108, No. 1-2, 05.2014, p. 3-29.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - The ignorant led by the blind
T2 - A hybrid human-machine vision system for fine-grained categorization
AU - Branson, Steve
AU - Van Horn, Grant
AU - Wah, Catherine
AU - Perona, Pietro
AU - Belongie, Serge
PY - 2014/5
Y1 - 2014/5
N2 - We present a visual recognition system for fine-grained visual categorization. The system is composed of a human and a machine working together and combines the complementary strengths of computer vision algorithms and (non-expert) human users. The human users provide two heterogeneous forms of information object part clicks and answers to multiple choice questions. The machine intelligently selects the most informative question to pose to the user in order to identify the object class as quickly as possible. By leveraging computer vision and analyzing the user responses, the overall amount of human effort required, measured in seconds, is minimized. Our formalism shows how to incorporate many different types of computer vision algorithms into a human-in-the-loop framework, including standard multiclass methods, part-based methods, and localized multiclass and attribute methods. We explore our ideas by building a field guide for bird identification. The experimental results demonstrate the strength of combining ignorant humans with poor-sighted machines the hybrid system achieves quick and accurate bird identification on a dataset containing 200 bird species.
AB - We present a visual recognition system for fine-grained visual categorization. The system is composed of a human and a machine working together and combines the complementary strengths of computer vision algorithms and (non-expert) human users. The human users provide two heterogeneous forms of information object part clicks and answers to multiple choice questions. The machine intelligently selects the most informative question to pose to the user in order to identify the object class as quickly as possible. By leveraging computer vision and analyzing the user responses, the overall amount of human effort required, measured in seconds, is minimized. Our formalism shows how to incorporate many different types of computer vision algorithms into a human-in-the-loop framework, including standard multiclass methods, part-based methods, and localized multiclass and attribute methods. We explore our ideas by building a field guide for bird identification. The experimental results demonstrate the strength of combining ignorant humans with poor-sighted machines the hybrid system achieves quick and accurate bird identification on a dataset containing 200 bird species.
KW - Attributes
KW - Birds
KW - Crowdsourcing
KW - Deformable part models
KW - Fine-grained categorization
KW - Human-in-the-loop
KW - Information gain
KW - Interactive
KW - Object recognition
KW - Parts
KW - Pose mixture models
UR - http://www.scopus.com/inward/record.url?scp=84900864212&partnerID=8YFLogxK
U2 - 10.1007/s11263-014-0698-4
DO - 10.1007/s11263-014-0698-4
M3 - Journal article
AN - SCOPUS:84900864212
VL - 108
SP - 3
EP - 29
JO - International Journal of Computer Vision
JF - International Journal of Computer Vision
SN - 0920-5691
IS - 1-2
ER -
ID: 302046045