Using computer vision of facial expressions to assess symptom domains and treatment response in antipsychotic-naïve patients with first-episode psychosis

Research output: Contribution to journalJournal articleResearchpeer-review

Background: Facial expressions are a core aspect of non-verbal communication. Reduced emotional expressiveness of the face is a common negative symptom of schizophrenia, however, quantifying negative symptoms can be clinically challenging and involves a considerable element of rater subjectivity. We used computer vision to investigate if (i) automated assessment of facial expressions captures negative as well as positive and general symptom domains, and (ii) if automated assessments are associated with treatment response in initially antipsychotic-naïve patients with first-episode psychosis. Method: We included 46 patients (mean age 25.4 (6.1); 65.2% males). Psychopathology was assessed at baseline and after 6 weeks of monotherapy with amisulpride using the Positive and Negative Syndrome Scale (PANSS). Baseline interview videos were recorded. Seventeen facial action units (AUs), that is, activation of muscles, from the Facial Action Coding System were extracted using OpenFace 2.0. A correlation matrix was calculated for each patient. Facial expressions were identified using spectral clustering at group-level. Associations between facial expressions and psychopathology were investigated using multiple linear regression. Results: Three clusters of facial expressions were identified related to different locations of the face. Cluster 1 was associated with positive and general symptoms at baseline, Cluster 2 was associated with all symptom domains, showing the strongest association with the negative domain, and Cluster 3 was only associated with general symptoms. Cluster 1 was significantly associated with the clinically rated improvement in positive and general symptoms after treatment, and Cluster 2 was significantly associated with clinical improvement in all domains. Conclusion: Using automated computer vision of facial expressions during PANSS interviews did not only capture negative symptoms but also combinations of the three overall domains of psychopathology. Moreover, automated assessments of facial expressions at baseline were associated with initial antipsychotic treatment response. The findings underscore the clinical relevance of facial expressions and motivate further investigations of computer vision in clinical psychiatry.

Original languageEnglish
JournalActa Psychiatrica Scandinavica
Number of pages10
ISSN0001-690X
DOIs
Publication statusE-pub ahead of print - 2024

Bibliographical note

Publisher Copyright:
© 2024 The Author(s). Acta Psychiatrica Scandinavica published by John Wiley & Sons Ltd.

    Research areas

  • action units, drug-naïve, facial expressions, psychopathology, schizophrenia

ID: 401972498