Biomedical image analysis competitions: The state of current participation practice
You are an author of this paper, . Is this page accurate?
You approved this page.
Thanks. We will fix this and let you know.
Lists
Only you see this
Abstract
The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.
BibTeX
@misc{eisenmann2022biomedical,
author = {Matthias Eisenmann and Annika Reinke and Vivienn Weru and Minu Dietlinde Tizabi and Fabian Isensee and Tim J. Adler and Patrick Godau and Veronika Cheplygina and Michal Kozubek and Sharib Ali and Anubha Gupta and Jan Kybic and Alison Noble and Carlos Ortiz de Solórzano and Samiksha Pachade and Caroline Petitjean and Daniel Sage and Donglai Wei and Elizabeth Wilden and Deepak Alapatt and Vincent Andrearczyk and Ujjwal Baid and Spyridon Bakas and Niranjan Balu and Sophia Bano and Vivek Singh Bawa and Jorge Bernal and Sebastian Bodenstedt and Alessandro Casella and Jinwook Choi and Olivier Commowick and Marie Daum and Adrien Depeursinge and Reuben Dorent and Jan Egger and Hannah Eichhorn and Sandy Engelhardt and Melanie Ganz and Gabriel Girard and Lasse Hansen and Mattias Heinrich and Nicholas Heller and Alessa Hering and Arnaud Huaulmé and Hyunjeong Kim and Bennett Landman and Hongwei Bran Li and Jianning Li and Jun Ma and Anne Martel and Carlos Martín-Isla and Bjoern Menze and Chinedu Innocent Nwoye and Valentin Oreiller and Nicolas Padoy and Sarthak Pati and Kelly Payette and Carole Sudre and Kimberlin van Wijnen and Armine Vardazaryan and Tom Vercauteren and Martin Wagner and Chuanbo Wang and Moi Hoon Yap and Zeyun Yu and Chun Yuan and Maximilian Zenk and Aneeq Zia and David Zimmerer and Rina Bao and Chanyeol Choi and Andrew Cohen and Oleh Dzyubachyk and Adrian Galdran and Tianyuan Gan and Tianqi Guo and Pradyumna Gupta and Mahmood Haithami and Edward Ho and Ikbeom Jang and Zhili Li and Zhengbo Luo and Filip Lux and Sokratis Makrogiannis and Dominik Müller and Young-tack Oh and Subeen Pang and Constantin Pape and Gorkem Polat and Charlotte Rosalie Reed and Kanghyun Ryu and Tim Scherr and Vajira Thambawita and Haoyu Wang and Xinliang Wang and Kele Xu and Hung Yeh and Doyeob Yeo and Yixuan Yuan and Yan Zeng and Xin Zhao and Julian Abbing and Jannes Adam and Nagesh Adluru and Niklas Agethen and Salman Ahmed and Yasmina Al Khalil and Mireia Alenyà and Esa Alhoniemi and Chengyang An and Talha Anwar and Tewodros Weldebirhan Arega and Netanell Avisdris and Dogu Baran Aydogan and Yingbin Bai and Maria Baldeon Calisto and Berke Doga Basaran and Marcel Beetz and Cheng Bian and Hao Bian and Kevin Blansit and Louise Bloch and Robert Bohnsack and Sara Bosticardo and Jack Breen and Mikael Brudfors and Raphael Brüngel and Mariano Cabezas and Alberto Cacciola and Zhiwei Chen and Yucong Chen and Daniel Tianming Chen and Minjeong Cho and Min-Kook Choi and Chuantao Xie Chuantao Xie and Dana Cobzas and Julien Cohen-Adad and Jorge Corral Acero and Sujit Kumar Das and Marcela de Oliveira and Hanqiu Deng and Guiming Dong and Lars Doorenbos and Cory Efird and Sergio Escalera and Di Fan and Mehdi Fatan Serj and Alexandre Fenneteau and Lucas Fidon and Patryk Filipiak and René Finzel and Nuno R. Freitas and Christoph M. Friedrich and Mitchell Fulton and Finn Gaida and Francesco Galati and Christoforos Galazis and Chang Hee Gan and Zheyao Gao and Shengbo Gao and Matej Gazda and Beerend Gerats and Neil Getty and Adam Gibicar and Ryan Gifford and Sajan Gohil and Maria Grammatikopoulou and Daniel Grzech and Orhun Güley and Timo Günnemann and Chunxu Guo and Sylvain Guy and Heonjin Ha and Luyi Han and Il Song Han and Ali Hatamizadeh and Tian He and Jimin Heo and Sebastian Hitziger and SeulGi Hong and SeungBum Hong and Rian Huang and Ziyan Huang and Markus Huellebrand and Stephan Huschauer and Mustaffa Hussain and Tomoo Inubushi and Ece Isik Polat and Mojtaba Jafaritadi and SeongHun Jeong and Bailiang Jian and Yuanhong Jiang and Zhifan Jiang and Yueming Jin and Smriti Joshi and Abdolrahim Kadkhodamohammadi and Reda Abdellah Kamraoui and Inha Kang and Junghwa Kang and Davood Karimi and April Khademi and Muhammad Irfan Khan and Suleiman A. Khan and Rishab Khantwal and Kwang-Ju Kim and Timothy Kline and Satoshi Kondo and Elina Kontio and Adrian Krenzer and Artem Kroviakov and Hugo Kuijf and Satyadwyoom Kumar and Francesco La Rosa and Abhi Lad and Doohee Lee and Minho Lee and Chiara Lena and Hao Li and Ling Li and Xingyu Li and Fuyuan Liao and KuanLun Liao and Arlindo Limede Oliveira and Chaonan Lin and Shan Lin and Akis Linardos and Marius George Linguraru and Han Liu and Tao Liu and Di Liu and Yanling Liu and João Lourenço-Silva and Jingpei Lu and Jiangshan Lu and Imanol Luengo and Christina B. Lund and Huan Minh Luu and Yi Lv and Yi Lv and Uzay Macar and Leon Maechler and Sina Mansour L. and Kenji Marshall and Moona Mazher and Richard McKinley and Alfonso Medela and Felix Meissen and Mingyuan Meng and Dylan Miller and Seyed Hossein Mirjahanmardi and Arnab Mishra and Samir Mitha and Hassan Mohy-ud-Din and Tony Chi Wing Mok and Gowtham Krishnan Murugesan and Enamundram Naga Karthik and Sahil Nalawade and Jakub Nalepa and Mohamed Naser and Ramin Nateghi and Hammad Naveed and Quang-Minh Nguyen and Cuong Nguyen Quoc and Brennan Nichyporuk and Bruno Oliveira and David Owen and Jimut Bahan Pal and Junwen Pan and Wentao Pan and Winnie Pang and Bogyu Park and Vivek Pawar and Kamlesh Pawar and Michael Peven and Lena Philipp and Tomasz Pieciak and Szymon Plotka and Marcel Plutat and Fattaneh Pourakpour and Domen Preložnik and Kumaradevan Punithakumar and Abdul Qayyum and Sandro Queirós and Arman Rahmim and Salar Razavi and Jintao Ren and Mina Rezaei and Jonathan Adam Rico and ZunHyan Rieu and Markus Rink and Johannes Roth and Yusely Ruiz-Gonzalez and Numan Saeed and Anindo Saha and Mostafa Salem and Ricardo Sanchez-Matilla and Kurt Schilling and Wei Shao and Zhiqiang Shen and Ruize Shi and Pengcheng Shi and Daniel Sobotka and Théodore Soulier and Bella Specktor Fadida and Danail Stoyanov and Timothy Sum Hon Mun and Xiaowu Sun and Rong Tao and Franz Thaler and Antoine Théberge and Felix Thielke and Helena Torres and Kareem A. Wahid and Jiacheng Wang and YiFei Wang and Wei Wang and Xiong Wang and Jianhui Wen and Ning Wen and Marek Wodzinski and Ye Wu and Fangfang Xia and Tianqi Xiang and Chen Xiaofei and Lizhan Xu and Tingting Xue and Yuxuan Yang and Lin Yang and Kai Yao and Huifeng Yao and Amirsaeed Yazdani and Michael Yip and Hwanseung Yoo and Fereshteh Yousefirizi and Shunkai Yu and Lei Yu and Jonathan Zamora and Ramy Ashraf Zeineldin and Dewen Zeng and Jianpeng Zhang and Bokai Zhang and Jiapeng Zhang and Fan Zhang and Huahong Zhang and Zhongchen Zhao and Zixuan Zhao and Jiachen Zhao and Can Zhao and Qingshuo Zheng and Yuheng Zhi and Ziqi Zhou and Baosheng Zou and Klaus Maier-Hein and Paul F. Jäger and Annette Kopp-Schneider and Lena Maier-Hein},
title = {{Biomedical image analysis competitions: The state of current participation practice}},
year = {2022},
eprint = {2212.08568},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2212.08568},
}Loading related content...