Efficient Training of Probabilistic Neural Networks for Survival Analysis

Christian Marius Lillelund; Martin Magris; Christian Fischer Pedersen

doi:10.1109/JBHI.2024.3417369

Efficient Training of Probabilistic Neural Networks for Survival Analysis

Christian Marius Lillelund, Martin Magris, Christian Fischer Pedersen

Institut for Elektro- og Computerteknologi - Signal Processing and Machine learning - Edison

Publikation: Bidrag til tidsskrift/Konferencebidrag i tidsskrift /Bidrag til avis › Tidsskriftartikel › Forskning › peer review

1 Citationer (Scopus)

Abstract

Variational Inference (VI) is a commonly used technique for approximate Bayesian inference and uncertainty estimation in deep learning models, yet it comes at a computational cost, as it doubles the number of trainable parameters to represent uncertainty. This rapidly becomes challenging in high-dimensional settings and motivates the use of alternative techniques for inference, such as Monte Carlo Dropout (MCD) or Spectral-normalized Neural Gaussian Process (SNGP). However, such methods have seen little adoption in survival analysis, and VI remains the prevalent approach for training probabilistic neural networks. In this paper, we investigate how to train deep probabilistic survival models in large datasets without introducing additional overhead in model complexity. To achieve this, we adopt three probabilistic approaches, namely VI, MCD, and SNGP, and evaluate them in terms of their prediction performance, calibration performance, and model complexity. In the context of probabilistic survival analysis, we investigate whether non-VI techniques can offer comparable or possibly improved prediction performance and uncertainty calibration compared to VI. In the MIMIC-IV dataset, we find that MCD aligns with VI in terms of the concordance index (0.748 vs. 0.743) and mean absolute error (254.9 vs. 254.7) using hinge loss, while providing C-calibrated uncertainty estimates. Moreover, our SNGP implementation provides D-calibrated survival functions in all datasets compared to VI (4/4 vs. 2/4, respectively). Our work encourages the use of techniques alternative to VI for survival analysis in high-dimensional datasets, where computational efficiency and overhead are of concern.

Originalsprog	Engelsk
Tidsskrift	IEEE Journal of Biomedical and Health Informatics
Vol/bind	PP
Sider (fra-til)	1-10
Antal sider	10
ISSN	2168-2194
DOI	https://doi.org/10.1109/JBHI.2024.3417369
Status	E-pub / Early view - 21 jun. 2024

Adgang til dokumentet

10.1109/JBHI.2024.3417369

Andre filer og links

Link to publication in Scopus

Citationsformater

@article{0e81dd9419df400c88b24f9daabcb2e6,

title = "Efficient Training of Probabilistic Neural Networks for Survival Analysis",

abstract = "Variational Inference (VI) is a commonly used technique for approximate Bayesian inference and uncertainty estimation in deep learning models, yet it comes at a computational cost, as it doubles the number of trainable parameters to represent uncertainty. This rapidly becomes challenging in high-dimensional settings and motivates the use of alternative techniques for inference, such as Monte Carlo Dropout (MCD) or Spectral-normalized Neural Gaussian Process (SNGP). However, such methods have seen little adoption in survival analysis, and VI remains the prevalent approach for training probabilistic neural networks. In this paper, we investigate how to train deep probabilistic survival models in large datasets without introducing additional overhead in model complexity. To achieve this, we adopt three probabilistic approaches, namely VI, MCD, and SNGP, and evaluate them in terms of their prediction performance, calibration performance, and model complexity. In the context of probabilistic survival analysis, we investigate whether non-VI techniques can offer comparable or possibly improved prediction performance and uncertainty calibration compared to VI. In the MIMIC-IV dataset, we find that MCD aligns with VI in terms of the concordance index (0.748 vs. 0.743) and mean absolute error (254.9 vs. 254.7) using hinge loss, while providing C-calibrated uncertainty estimates. Moreover, our SNGP implementation provides D-calibrated survival functions in all datasets compared to VI (4/4 vs. 2/4, respectively). Our work encourages the use of techniques alternative to VI for survival analysis in high-dimensional datasets, where computational efficiency and overhead are of concern.",

keywords = "Analytical models, Bayes methods, Bayesian learning, Computational modeling, Hazards, Machine learning, neural networks, Predictive models, Probabilistic logic, survival analysis, Uncertainty, uncertainty estimation",

author = "Lillelund, {Christian Marius} and Martin Magris and Pedersen, {Christian Fischer}",

note = "Publisher Copyright: IEEE",

year = "2024",

month = jun,

day = "21",

doi = "10.1109/JBHI.2024.3417369",

language = "English",

volume = "PP",

pages = "1--10",

journal = "IEEE Journal of Biomedical and Health Informatics",

issn = "2168-2194",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Efficient Training of Probabilistic Neural Networks for Survival Analysis

AU - Lillelund, Christian Marius

AU - Magris, Martin

AU - Pedersen, Christian Fischer

N1 - Publisher Copyright: IEEE

PY - 2024/6/21

Y1 - 2024/6/21

N2 - Variational Inference (VI) is a commonly used technique for approximate Bayesian inference and uncertainty estimation in deep learning models, yet it comes at a computational cost, as it doubles the number of trainable parameters to represent uncertainty. This rapidly becomes challenging in high-dimensional settings and motivates the use of alternative techniques for inference, such as Monte Carlo Dropout (MCD) or Spectral-normalized Neural Gaussian Process (SNGP). However, such methods have seen little adoption in survival analysis, and VI remains the prevalent approach for training probabilistic neural networks. In this paper, we investigate how to train deep probabilistic survival models in large datasets without introducing additional overhead in model complexity. To achieve this, we adopt three probabilistic approaches, namely VI, MCD, and SNGP, and evaluate them in terms of their prediction performance, calibration performance, and model complexity. In the context of probabilistic survival analysis, we investigate whether non-VI techniques can offer comparable or possibly improved prediction performance and uncertainty calibration compared to VI. In the MIMIC-IV dataset, we find that MCD aligns with VI in terms of the concordance index (0.748 vs. 0.743) and mean absolute error (254.9 vs. 254.7) using hinge loss, while providing C-calibrated uncertainty estimates. Moreover, our SNGP implementation provides D-calibrated survival functions in all datasets compared to VI (4/4 vs. 2/4, respectively). Our work encourages the use of techniques alternative to VI for survival analysis in high-dimensional datasets, where computational efficiency and overhead are of concern.

AB - Variational Inference (VI) is a commonly used technique for approximate Bayesian inference and uncertainty estimation in deep learning models, yet it comes at a computational cost, as it doubles the number of trainable parameters to represent uncertainty. This rapidly becomes challenging in high-dimensional settings and motivates the use of alternative techniques for inference, such as Monte Carlo Dropout (MCD) or Spectral-normalized Neural Gaussian Process (SNGP). However, such methods have seen little adoption in survival analysis, and VI remains the prevalent approach for training probabilistic neural networks. In this paper, we investigate how to train deep probabilistic survival models in large datasets without introducing additional overhead in model complexity. To achieve this, we adopt three probabilistic approaches, namely VI, MCD, and SNGP, and evaluate them in terms of their prediction performance, calibration performance, and model complexity. In the context of probabilistic survival analysis, we investigate whether non-VI techniques can offer comparable or possibly improved prediction performance and uncertainty calibration compared to VI. In the MIMIC-IV dataset, we find that MCD aligns with VI in terms of the concordance index (0.748 vs. 0.743) and mean absolute error (254.9 vs. 254.7) using hinge loss, while providing C-calibrated uncertainty estimates. Moreover, our SNGP implementation provides D-calibrated survival functions in all datasets compared to VI (4/4 vs. 2/4, respectively). Our work encourages the use of techniques alternative to VI for survival analysis in high-dimensional datasets, where computational efficiency and overhead are of concern.

KW - Analytical models

KW - Bayes methods

KW - Bayesian learning

KW - Computational modeling

KW - Hazards

KW - Machine learning

KW - neural networks

KW - Predictive models

KW - Probabilistic logic

KW - survival analysis

KW - Uncertainty

KW - uncertainty estimation

UR - http://www.scopus.com/inward/record.url?scp=85196718109&partnerID=8YFLogxK

U2 - 10.1109/JBHI.2024.3417369

DO - 10.1109/JBHI.2024.3417369

M3 - Journal article

C2 - 38905091

AN - SCOPUS:85196718109

SN - 2168-2194

VL - PP

SP - 1

EP - 10

JO - IEEE Journal of Biomedical and Health Informatics

JF - IEEE Journal of Biomedical and Health Informatics

ER -

Efficient Training of Probabilistic Neural Networks for Survival Analysis

Abstract

Adgang til dokumentet

Andre filer og links

Fingeraftryk

Citationsformater