Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Final published version, 177 KB, PDF document
Counterfactually Augmented Data (CAD) aims to improve out-of-domain generalizability, an indicator of model robustness. The improvement is credited to promoting core features of the construct over spurious artifacts that happen to correlate with it. Yet, over-relying on core features may lead to unintended model bias. Especially, construct-driven CAD-perturbations of core features-may induce models to ignore the context in which core features are used. Here, we test models for sexism and hate speech detection on challenging data: non-hateful and nonsexist usage of identity and gendered terms. On these hard cases, models trained on CAD, especially construct-driven CAD, show higher false positive rates than models trained on the original, unperturbed data. Using a diverse set of CAD-construct-driven and construct-agnostic-reduces such unintended bias.
|Title of host publication||Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies|
|Publisher||Association for Computational Linguistics (ACL)|
|Publication status||Published - 2022|
|Event||2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022 - Seattle, United States|
Duration: 10 Jul 2022 → 15 Jul 2022
|Conference||2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022|
|Periode||10/07/2022 → 15/07/2022|
|Sponsor||Amazon, Bloomberg, et al., Google Research, LIVE PERSON, Meta|
© 2022 Association for Computational Linguistics.