Automatic sleep scoring using patient-specific ensemble models and knowledge distillation for ear-EEG data

TY - JOUR

T1 - Automatic sleep scoring using patient-specific ensemble models and knowledge distillation for ear-EEG data

AU - Borup, Kenneth

AU - Kidmose, Preben

AU - Phan, Huy

AU - Mikkelsen, Kaare

PY - 2023/3

Y1 - 2023/3

N2 - Human sleep can be described as a series of transitions between distinct states. This makes automatic sleep analysis (scoring) suitable for an automatic implementation using machine learning. However, the task becomes harder when data is sampled using more light-weight or mobile equipment, often chosen due to greater comfort for the patient. In this study we investigate the improvement in sleep scoring when multiple state-of-the-art neural networks are joined into an ensemble, and subsequently distilled into a single model of identical network architecture, but with improved predictive performance. In this study we investigate ensembles of up to 10 networks, and show that, on the same data, ensembles of neural networks perform better than each single subject model (improvement: 2.4 and that this improvement can be transferred back into a single network using a combination of patient specific data and knowledge distillation. The study demonstrates both a way to further improve automatic sleep scoring from mobile devices, which in itself is interesting, but also highlights the great potential of the vast amounts of unlabeled personal data which will become available from personal recording devices.

AB - Human sleep can be described as a series of transitions between distinct states. This makes automatic sleep analysis (scoring) suitable for an automatic implementation using machine learning. However, the task becomes harder when data is sampled using more light-weight or mobile equipment, often chosen due to greater comfort for the patient. In this study we investigate the improvement in sleep scoring when multiple state-of-the-art neural networks are joined into an ensemble, and subsequently distilled into a single model of identical network architecture, but with improved predictive performance. In this study we investigate ensembles of up to 10 networks, and show that, on the same data, ensembles of neural networks perform better than each single subject model (improvement: 2.4 and that this improvement can be transferred back into a single network using a combination of patient specific data and knowledge distillation. The study demonstrates both a way to further improve automatic sleep scoring from mobile devices, which in itself is interesting, but also highlights the great potential of the vast amounts of unlabeled personal data which will become available from personal recording devices.

KW - Automatic sleep scoring

KW - Ensemble models

KW - Knowledge distillation

KW - Light-weight sleep scoring

KW - Personalized models

KW - Semi-supervised learning

UR - http://www.scopus.com/inward/record.url?scp=85144415184&partnerID=8YFLogxK

U2 - 10.1016/j.bspc.2022.104496

DO - 10.1016/j.bspc.2022.104496

M3 - Journal article

SN - 1746-8094

VL - 81

JO - Biomedical Signal Processing and Control

JF - Biomedical Signal Processing and Control

M1 - 104496

ER -