On Optimizing Human-Machine Task Assignments
Research output: Working paper › Preprint › Research
Standard
On Optimizing Human-Machine Task Assignments. / Belongie, Serge; Veit, Andreas; Wilber, Michael J.; Vaish, Rajan; Davis, James; Anand, Vishal; Aviral, Anshu; Chakrabarty, Prithvijit; Chandak, Yash; Chaturvedi, Sidharth; Devaraj, Chinmaya; Dhall, Ankit; Dwivedi, Utkarsh; Gupte, Sanket; Sridhar, Sharath N.; Paga, Karthik; Pahuja, Anuj; Raisinghani, Aditya; Sharma, Ayush; Sharma, Shweta; Sinha, Darpana; Thakkar, Nisarg; Vignesh, K. Bala; Verma, Utkarsh; Abhishek, Kanniganti; Agrawal, Amod; Aishwarya, Arya; Bhattacharjee, Aurgho; Dhanasekar, Sarveshwaran; Gullapalli, Venkata Karthik; Gupta, Shuchita; Chandana, G; Jain, Kinjal; Kapur, Simran; Kasula, Meghana; Kumar, Shashi; Kundaliya, Parth; Mathur, Utkarsh; Mishra, Alankrit; Mudgal, Aayush; Nadimpalli, Aditya; Nihit, Munakala Sree; Periwal, Akanksha; Sagar, Ayush; Shah, Ayush; Sharma, Vikas; Sharma, Yashovardhan; Siddiqui, Faizal; Singh, Virender; Abhinav, S.; Tambwekar, Pradyumna; Taskin, Rashida; Tripathi, Ankit; Yadav, Anurag D.
2015.Research output: Working paper › Preprint › Research
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - UNPB
T1 - On Optimizing Human-Machine Task Assignments
AU - Belongie, Serge
AU - Veit, Andreas
AU - Wilber, Michael J.
AU - Vaish, Rajan
AU - Davis, James
AU - Anand, Vishal
AU - Aviral, Anshu
AU - Chakrabarty, Prithvijit
AU - Chandak, Yash
AU - Chaturvedi, Sidharth
AU - Devaraj, Chinmaya
AU - Dhall, Ankit
AU - Dwivedi, Utkarsh
AU - Gupte, Sanket
AU - Sridhar, Sharath N.
AU - Paga, Karthik
AU - Pahuja, Anuj
AU - Raisinghani, Aditya
AU - Sharma, Ayush
AU - Sharma, Shweta
AU - Sinha, Darpana
AU - Thakkar, Nisarg
AU - Vignesh, K. Bala
AU - Verma, Utkarsh
AU - Abhishek, Kanniganti
AU - Agrawal, Amod
AU - Aishwarya, Arya
AU - Bhattacharjee, Aurgho
AU - Dhanasekar, Sarveshwaran
AU - Gullapalli, Venkata Karthik
AU - Gupta, Shuchita
AU - Chandana, G
AU - Jain, Kinjal
AU - Kapur, Simran
AU - Kasula, Meghana
AU - Kumar, Shashi
AU - Kundaliya, Parth
AU - Mathur, Utkarsh
AU - Mishra, Alankrit
AU - Mudgal, Aayush
AU - Nadimpalli, Aditya
AU - Nihit, Munakala Sree
AU - Periwal, Akanksha
AU - Sagar, Ayush
AU - Shah, Ayush
AU - Sharma, Vikas
AU - Sharma, Yashovardhan
AU - Siddiqui, Faizal
AU - Singh, Virender
AU - Abhinav, S.
AU - Tambwekar, Pradyumna
AU - Taskin, Rashida
AU - Tripathi, Ankit
AU - Yadav, Anurag D.
PY - 2015/9/24
Y1 - 2015/9/24
N2 - When crowdsourcing systems are used in combination with machine inference systems in the real world, they benefit the most when the machine system is deeply integrated with the crowd workers. However, if researchers wish to integrate the crowd with "off-the-shelf" machine classifiers, this deep integration is not always possible. This work explores two strategies to increase accuracy and decrease cost under this setting. First, we show that reordering tasks presented to the human can create a significant accuracy improvement. Further, we show that greedily choosing parameters to maximize machine accuracy is sub-optimal, and joint optimization of the combined system improves performance.
AB - When crowdsourcing systems are used in combination with machine inference systems in the real world, they benefit the most when the machine system is deeply integrated with the crowd workers. However, if researchers wish to integrate the crowd with "off-the-shelf" machine classifiers, this deep integration is not always possible. This work explores two strategies to increase accuracy and decrease cost under this setting. First, we show that reordering tasks presented to the human can create a significant accuracy improvement. Further, we show that greedily choosing parameters to maximize machine accuracy is sub-optimal, and joint optimization of the combined system improves performance.
UR - https://vision.cornell.edu/se3/wp-content/uploads/2015/10/1509.07543v1.pdf
U2 - https://doi.org/10.48550/arXiv.1509.07543
DO - https://doi.org/10.48550/arXiv.1509.07543
M3 - Preprint
BT - On Optimizing Human-Machine Task Assignments
ER -
ID: 307530275