TY - JOUR
T1 - Enhancing the reliability of deep learning-based head and neck tumour segmentation using uncertainty estimation with multi-modal images
AU - Ren, Jintao
AU - Teuwen, Jonas
AU - Nijkamp, Jasper
AU - Rasmussen, Mathis
AU - Gouw, Zeno
AU - Grau Eriksen, Jesper
AU - Sonke, Jan Jakob
AU - Korreman, Stine
N1 - Publisher Copyright:
© 2024 Institute of Physics and Engineering in Medicine.
PY - 2024/8/5
Y1 - 2024/8/5
N2 - Objective. Deep learning shows promise in autosegmentation of head and neck cancer (HNC) primary tumours (GTV-T) and nodal metastases (GTV-N). However, errors such as including non-tumour regions or missing nodal metastases still occur. Conventional methods often make overconfident predictions, compromising reliability. Incorporating uncertainty estimation, which provides calibrated confidence intervals can address this issue. Our aim was to investigate the efficacy of various uncertainty estimation methods in improving segmentation reliability. We evaluated their confidence levels in voxel predictions and ability to reveal potential segmentation errors. Approach. We retrospectively collected data from 567 HNC patients with diverse cancer sites and multi-modality images (CT, PET, T1-, and T2-weighted MRI) along with their clinical GTV-T/N delineations. Using the nnUNet 3D segmentation pipeline, we compared seven uncertainty estimation methods, evaluating them based on segmentation accuracy (Dice similarity coefficient, DSC), confidence calibration (Expected Calibration Error, ECE), and their ability to reveal segmentation errors (Uncertainty-Error overlap using DSC, UE-DSC). Main results. Evaluated on the hold-out test dataset (n = 97), the median DSC scores for GTV-T and GTV-N segmentation across all uncertainty estimation methods had a narrow range, from 0.73 to 0.76 and 0.78 to 0.80, respectively. In contrast, the median ECE exhibited a wider range, from 0.30 to 0.12 for GTV-T and 0.25 to 0.09 for GTV-N. Similarly, the median UE-DSC also ranged broadly, from 0.21 to 0.38 for GTV-T and 0.22 to 0.36 for GTV-N. A probabilistic network—PhiSeg method consistently demonstrated the best performance in terms of ECE and UE-DSC. Significance. Our study highlights the importance of uncertainty estimation in enhancing the reliability of deep learning for autosegmentation of HNC GTV. The results show that while segmentation accuracy can be similar across methods, their reliability, measured by calibration error and uncertainty-error overlap, varies significantly. Used with visualisation maps, these methods may effectively pinpoint uncertainties and potential errors at the voxel level.
AB - Objective. Deep learning shows promise in autosegmentation of head and neck cancer (HNC) primary tumours (GTV-T) and nodal metastases (GTV-N). However, errors such as including non-tumour regions or missing nodal metastases still occur. Conventional methods often make overconfident predictions, compromising reliability. Incorporating uncertainty estimation, which provides calibrated confidence intervals can address this issue. Our aim was to investigate the efficacy of various uncertainty estimation methods in improving segmentation reliability. We evaluated their confidence levels in voxel predictions and ability to reveal potential segmentation errors. Approach. We retrospectively collected data from 567 HNC patients with diverse cancer sites and multi-modality images (CT, PET, T1-, and T2-weighted MRI) along with their clinical GTV-T/N delineations. Using the nnUNet 3D segmentation pipeline, we compared seven uncertainty estimation methods, evaluating them based on segmentation accuracy (Dice similarity coefficient, DSC), confidence calibration (Expected Calibration Error, ECE), and their ability to reveal segmentation errors (Uncertainty-Error overlap using DSC, UE-DSC). Main results. Evaluated on the hold-out test dataset (n = 97), the median DSC scores for GTV-T and GTV-N segmentation across all uncertainty estimation methods had a narrow range, from 0.73 to 0.76 and 0.78 to 0.80, respectively. In contrast, the median ECE exhibited a wider range, from 0.30 to 0.12 for GTV-T and 0.25 to 0.09 for GTV-N. Similarly, the median UE-DSC also ranged broadly, from 0.21 to 0.38 for GTV-T and 0.22 to 0.36 for GTV-N. A probabilistic network—PhiSeg method consistently demonstrated the best performance in terms of ECE and UE-DSC. Significance. Our study highlights the importance of uncertainty estimation in enhancing the reliability of deep learning for autosegmentation of HNC GTV. The results show that while segmentation accuracy can be similar across methods, their reliability, measured by calibration error and uncertainty-error overlap, varies significantly. Used with visualisation maps, these methods may effectively pinpoint uncertainties and potential errors at the voxel level.
KW - deep learning
KW - gross tumour volume
KW - head and neck cancer
KW - radiotherapy
KW - tumour segmentation
KW - uncertainty estimation
KW - uncertainty quantification
UR - http://www.scopus.com/inward/record.url?scp=85200623040&partnerID=8YFLogxK
U2 - 10.1088/1361-6560/ad682d
DO - 10.1088/1361-6560/ad682d
M3 - Journal article
C2 - 39059432
AN - SCOPUS:85200623040
SN - 0031-9155
VL - 69
JO - Physics in Medicine and Biology
JF - Physics in Medicine and Biology
IS - 16
M1 - 165018
ER -