An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text. / Kementchedjhieva, Yova; Chalkidis, Ilias.
Findings of the Association for Computational Linguistics, ACL 2023. Association for Computational Linguistics (ACL), 2023. p. 5828-5843 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text
AU - Kementchedjhieva, Yova
AU - Chalkidis, Ilias
N1 - Publisher Copyright: © 2023 Association for Computational Linguistics.
PY - 2023
Y1 - 2023
N2 - Standard methods for multi-label text classification largely rely on encoder-only pre-trained language models, whereas encoder-decoder models have proven more effective in other classification tasks. In this study, we compare four methods for multi-label classification, two based on an encoder only, and two based on an encoder-decoder. We carry out experiments on four datasets-two in the legal domain and two in the biomedical domain, each with two levels of label granularity- and always depart from the same pre-trained model, T5. Our results show that encoder-decoder methods outperform encoder-only methods, with a growing advantage on more complex datasets and labeling schemes of finer granularity. Using encoder-decoder models in a non-autoregressive fashion, in particular, yields the best performance overall, so we further study this approach through ablations to better understand its strengths.
AB - Standard methods for multi-label text classification largely rely on encoder-only pre-trained language models, whereas encoder-decoder models have proven more effective in other classification tasks. In this study, we compare four methods for multi-label classification, two based on an encoder only, and two based on an encoder-decoder. We carry out experiments on four datasets-two in the legal domain and two in the biomedical domain, each with two levels of label granularity- and always depart from the same pre-trained model, T5. Our results show that encoder-decoder methods outperform encoder-only methods, with a growing advantage on more complex datasets and labeling schemes of finer granularity. Using encoder-decoder models in a non-autoregressive fashion, in particular, yields the best performance overall, so we further study this approach through ablations to better understand its strengths.
UR - http://www.scopus.com/inward/record.url?scp=85174996579&partnerID=8YFLogxK
U2 - 10.18653/v1/2023.findings-acl.360
DO - 10.18653/v1/2023.findings-acl.360
M3 - Article in proceedings
AN - SCOPUS:85174996579
T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics
SP - 5828
EP - 5843
BT - Findings of the Association for Computational Linguistics, ACL 2023
PB - Association for Computational Linguistics (ACL)
T2 - 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023
Y2 - 9 July 2023 through 14 July 2023
ER -
ID: 374650746