The Liver Tumor Segmentation Benchmark (LiTS)
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
The Liver Tumor Segmentation Benchmark (LiTS). / Bilic, Patrick; Christ, Patrick; Li, Hongwei Bran; Vorontsov, Eugene; Ben-Cohen, Avi; Kaissis, Georgios; Szeskin, Adi; Jacobs, Colin; Mamani, Gabriel Efrain Humpire; Chartrand, Gabriel; Lohöfer, Fabian; Holch, Julian Walter; Sommer, Wieland; Hofmann, Felix; Hostettler, Alexandre; Lev-Cohain, Naama; Drozdzal, Michal; Amitai, Michal Marianne; Vivanti, Refael; Sosna, Jacob; Ezhov, Ivan; Sekuboyina, Anjany; Navarro, Fernando; Kofler, Florian; Paetzold, Johannes C; Shit, Suprosanna; Hu, Xiaobin; Lipková, Jana; Rempfler, Markus; Piraud, Marie; Kirschke, Jan; Wiestler, Benedikt; Zhang, Zhiheng; Hülsemeyer, Christian; Beetz, Marcel; Ettlinger, Florian; Antonelli, Michela; Bae, Woong; Bellver, Míriam; Bi, Lei; Chen, Hao; Chlebus, Grzegorz; Dam, Erik B; Dou, Qi; Fu, Chi-Wing; Georgescu, Bogdan; Giró-I-Nieto, Xavier; Gruen, Felix; Han, Xu; Heng, Pheng-Ann; Hesser, Jürgen; Moltz, Jan Hendrik; Igel, Christian; Isensee, Fabian; Jäger, Paul; Jia, Fucang; Kaluva, Krishna Chaitanya; Khened, Mahendra; Kim, Ildoo; Kim, Jae-Hun; Kim, Sungwoong; Kohl, Simon; Konopczynski, Tomasz; Kori, Avinash; Krishnamurthi, Ganapathy; Li, Fan; Li, Hongchao; Li, Junbo; Li, Xiaomeng; Lowengrub, John; Ma, Jun; Maier-Hein, Klaus; Maninis, Kevis-Kokitsi; Meine, Hans; Merhof, Dorit; Pai, Akshay; Perslev, Mathias; Petersen, Jens; Pont-Tuset, Jordi; Qi, Jin; Qi, Xiaojuan; Rippel, Oliver; Roth, Karsten; Sarasua, Ignacio; Schenk, Andrea; Shen, Zengming; Torres, Jordi; Wachinger, Christian; Wang, Chunliang; Weninger, Leon; Wu, Jianrong; Xu, Daguang; Yang, Xiaoping; Yu, Simon Chun-Ho; Yuan, Yading; Yue, Miao; Zhang, Liping; Cardoso, Jorge; Bakas, Spyridon; Braren, Rickmer; Heinemann, Volker; Pal, Christopher; Tang, An; Kadoury, Samuel; Soler, Luc; van Ginneken, Bram; Greenspan, Hayit; Joskowicz, Leo; Menze, Bjoern.
In: Medical Image Analysis, Vol. 84, 102680, 2023.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - The Liver Tumor Segmentation Benchmark (LiTS)
AU - Bilic, Patrick
AU - Christ, Patrick
AU - Li, Hongwei Bran
AU - Vorontsov, Eugene
AU - Ben-Cohen, Avi
AU - Kaissis, Georgios
AU - Szeskin, Adi
AU - Jacobs, Colin
AU - Mamani, Gabriel Efrain Humpire
AU - Chartrand, Gabriel
AU - Lohöfer, Fabian
AU - Holch, Julian Walter
AU - Sommer, Wieland
AU - Hofmann, Felix
AU - Hostettler, Alexandre
AU - Lev-Cohain, Naama
AU - Drozdzal, Michal
AU - Amitai, Michal Marianne
AU - Vivanti, Refael
AU - Sosna, Jacob
AU - Ezhov, Ivan
AU - Sekuboyina, Anjany
AU - Navarro, Fernando
AU - Kofler, Florian
AU - Paetzold, Johannes C
AU - Shit, Suprosanna
AU - Hu, Xiaobin
AU - Lipková, Jana
AU - Rempfler, Markus
AU - Piraud, Marie
AU - Kirschke, Jan
AU - Wiestler, Benedikt
AU - Zhang, Zhiheng
AU - Hülsemeyer, Christian
AU - Beetz, Marcel
AU - Ettlinger, Florian
AU - Antonelli, Michela
AU - Bae, Woong
AU - Bellver, Míriam
AU - Bi, Lei
AU - Chen, Hao
AU - Chlebus, Grzegorz
AU - Dam, Erik B
AU - Dou, Qi
AU - Fu, Chi-Wing
AU - Georgescu, Bogdan
AU - Giró-I-Nieto, Xavier
AU - Gruen, Felix
AU - Han, Xu
AU - Heng, Pheng-Ann
AU - Hesser, Jürgen
AU - Moltz, Jan Hendrik
AU - Igel, Christian
AU - Isensee, Fabian
AU - Jäger, Paul
AU - Jia, Fucang
AU - Kaluva, Krishna Chaitanya
AU - Khened, Mahendra
AU - Kim, Ildoo
AU - Kim, Jae-Hun
AU - Kim, Sungwoong
AU - Kohl, Simon
AU - Konopczynski, Tomasz
AU - Kori, Avinash
AU - Krishnamurthi, Ganapathy
AU - Li, Fan
AU - Li, Hongchao
AU - Li, Junbo
AU - Li, Xiaomeng
AU - Lowengrub, John
AU - Ma, Jun
AU - Maier-Hein, Klaus
AU - Maninis, Kevis-Kokitsi
AU - Meine, Hans
AU - Merhof, Dorit
AU - Pai, Akshay
AU - Perslev, Mathias
AU - Petersen, Jens
AU - Pont-Tuset, Jordi
AU - Qi, Jin
AU - Qi, Xiaojuan
AU - Rippel, Oliver
AU - Roth, Karsten
AU - Sarasua, Ignacio
AU - Schenk, Andrea
AU - Shen, Zengming
AU - Torres, Jordi
AU - Wachinger, Christian
AU - Wang, Chunliang
AU - Weninger, Leon
AU - Wu, Jianrong
AU - Xu, Daguang
AU - Yang, Xiaoping
AU - Yu, Simon Chun-Ho
AU - Yuan, Yading
AU - Yue, Miao
AU - Zhang, Liping
AU - Cardoso, Jorge
AU - Bakas, Spyridon
AU - Braren, Rickmer
AU - Heinemann, Volker
AU - Pal, Christopher
AU - Tang, An
AU - Kadoury, Samuel
AU - Soler, Luc
AU - van Ginneken, Bram
AU - Greenspan, Hayit
AU - Joskowicz, Leo
AU - Menze, Bjoern
N1 - Copyright © 2022 The Author(s). Published by Elsevier B.V. All rights reserved.
PY - 2023
Y1 - 2023
N2 - In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.
AB - In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.
U2 - 10.1016/j.media.2022.102680
DO - 10.1016/j.media.2022.102680
M3 - Journal article
C2 - 36481607
VL - 84
JO - Medical Image Analysis
JF - Medical Image Analysis
SN - 1361-8415
M1 - 102680
ER -
ID: 328434446