TY - ABST

T1 - Frustration as Emotional Stance: when blind people encounter problems with new technology

AU - Nielsen, Ann Merrit Rikke

AU - Due, Brian Lystgaard

PY - 2022

Y1 - 2022

N2 - Visually Impaired People (VIP) increasingly use mainstream technologies, such as smart phones and digital assistants featuring AI in their everyday lives. The VIP’s limited visual access can prove challenging when learning to use these technologies and this may lead to considerable frustration, observable in interaction as emotional displays. This paper examines atypical interaction between VIP and seeing participants both when VIP are being instructed in or testing the use of a Google Home assistant and when they are using the Seeing AI app for shopping for the first time. Both settings involve learning-by-doing (Lindwall & Ekström, 2012), a lot of trial-and-error sequences, and a great risk of experiencing being ‘put on the spot’. Applying multimodal EMCA (Streeck et al., 2011) to video recordings of these settings this paper explores VIP’s displayed emotional stance (Ruusuvuori, 2012; Peräkylä & Sorjonen, 2012) when partaking in these complex activities, specifically the different formations of the action type “frustration”. Frustration has been shown to be displayed within the situated activities of demonstrating and leaning (C. Goodwin, 2007), in academic feedback (Sandlund, 2004), and in mundane family interaction (M. H. Goodwin et al., 2012). We explore how VIP’s, via facial expression, prosody, gesture, explaining, accounting, outbursts, and explicit lexical formulations of own affectual state, produce a large number of multimodal emotional displays (C. Goodwin & Goodwin, 2000; Kaukomaa, 2015). Examining the sequential organization of these displays of frustration and demonstrating how co-participants orient (or not) to these features of the interaction as affective (Edwards, 1999) we uses these examples to discuss how increased sensibility to the VIPs reaction to being ‘put on the spot’ when performing complex technology-related tasks might improve the learning environment. Thus, this paper contributes to EMCA research in how to increase VIP’s safe inclusion in collaborative leaning activities, ultimately enhancing their self-reliance.Edwards, D. (1999). Emotion Discourse. Culture & Psychology, 5(3), 271–291. https://doi.org/10.1177/1354067X9953001Goodwin, C. (2007). Participation, Stance and Affect in the Organization of Activities. Discourse and society, 18(1), 53–74.Goodwin, C., & Goodwin, M. H. (2000). Emotion within Situated Activity. I Duranti (ed.) Linguistic Anthropology: A Reader. (s. 239–257). Malden, MA, Oxford, Blackwell.Goodwin, M. H., Cekaite, A., & Goodwin, C. (2012). Emotion as Stance. I Emotion in Interaction. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199730735.003.0002Goodwin, M. H., Cekaite, A., Goodwin, C., & Tulbert, E. (2011). Emotion as Stance. I A. Peräkylä & M-L. Sorjônen (eds.) Emotion in interaction. Oxford University Press.Lindwall, O., & Ekström, A. (2012). Instruction-in-Interaction: The Teaching and Learning of a Manual Skill. Human Studies, 35(1), 27–49. https://doi.org/10.1007/s10746-012-9213-5Local, J., & Walker, G. (2008). Stance and affect in conversation: On the interplay of sequential and phonetic resources. Text & Talk, 28(6), 723–747. https://doi.org/10.1515/TEXT.2008.037Peräkylä, A., & Sorjonen, M.-L. (2012). Emotion in Interaction (Illustrated edition). OUP USA.Ruusuvuori, J. (2012). Emotion, Affect and Conversation. I The Handbook of Conversation Analysis (s. 330–349). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781118325001.ch16Sandlund, E. (2004). Feeling by Doing. The social organization. . . [PhD dissertation].Streeck, J., Goodwin, C., & LeBaron, C. D. (2011). Embodied interaction: Language and body in the material world /. University Press.

AB - Visually Impaired People (VIP) increasingly use mainstream technologies, such as smart phones and digital assistants featuring AI in their everyday lives. The VIP’s limited visual access can prove challenging when learning to use these technologies and this may lead to considerable frustration, observable in interaction as emotional displays. This paper examines atypical interaction between VIP and seeing participants both when VIP are being instructed in or testing the use of a Google Home assistant and when they are using the Seeing AI app for shopping for the first time. Both settings involve learning-by-doing (Lindwall & Ekström, 2012), a lot of trial-and-error sequences, and a great risk of experiencing being ‘put on the spot’. Applying multimodal EMCA (Streeck et al., 2011) to video recordings of these settings this paper explores VIP’s displayed emotional stance (Ruusuvuori, 2012; Peräkylä & Sorjonen, 2012) when partaking in these complex activities, specifically the different formations of the action type “frustration”. Frustration has been shown to be displayed within the situated activities of demonstrating and leaning (C. Goodwin, 2007), in academic feedback (Sandlund, 2004), and in mundane family interaction (M. H. Goodwin et al., 2012). We explore how VIP’s, via facial expression, prosody, gesture, explaining, accounting, outbursts, and explicit lexical formulations of own affectual state, produce a large number of multimodal emotional displays (C. Goodwin & Goodwin, 2000; Kaukomaa, 2015). Examining the sequential organization of these displays of frustration and demonstrating how co-participants orient (or not) to these features of the interaction as affective (Edwards, 1999) we uses these examples to discuss how increased sensibility to the VIPs reaction to being ‘put on the spot’ when performing complex technology-related tasks might improve the learning environment. Thus, this paper contributes to EMCA research in how to increase VIP’s safe inclusion in collaborative leaning activities, ultimately enhancing their self-reliance.Edwards, D. (1999). Emotion Discourse. Culture & Psychology, 5(3), 271–291. https://doi.org/10.1177/1354067X9953001Goodwin, C. (2007). Participation, Stance and Affect in the Organization of Activities. Discourse and society, 18(1), 53–74.Goodwin, C., & Goodwin, M. H. (2000). Emotion within Situated Activity. I Duranti (ed.) Linguistic Anthropology: A Reader. (s. 239–257). Malden, MA, Oxford, Blackwell.Goodwin, M. H., Cekaite, A., & Goodwin, C. (2012). Emotion as Stance. I Emotion in Interaction. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199730735.003.0002Goodwin, M. H., Cekaite, A., Goodwin, C., & Tulbert, E. (2011). Emotion as Stance. I A. Peräkylä & M-L. Sorjônen (eds.) Emotion in interaction. Oxford University Press.Lindwall, O., & Ekström, A. (2012). Instruction-in-Interaction: The Teaching and Learning of a Manual Skill. Human Studies, 35(1), 27–49. https://doi.org/10.1007/s10746-012-9213-5Local, J., & Walker, G. (2008). Stance and affect in conversation: On the interplay of sequential and phonetic resources. Text & Talk, 28(6), 723–747. https://doi.org/10.1515/TEXT.2008.037Peräkylä, A., & Sorjonen, M.-L. (2012). Emotion in Interaction (Illustrated edition). OUP USA.Ruusuvuori, J. (2012). Emotion, Affect and Conversation. I The Handbook of Conversation Analysis (s. 330–349). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781118325001.ch16Sandlund, E. (2004). Feeling by Doing. The social organization. . . [PhD dissertation].Streeck, J., Goodwin, C., & LeBaron, C. D. (2011). Embodied interaction: Language and body in the material world /. University Press.

UR - https://conferences.ncl.ac.uk/aic2022/programme/

M3 - Conference abstract for conference

Y2 - 27 June 2022 through 29 June 2022

ER -