Automatic sleep scoring using patient-specific ensemble models and knowledge distillation for ear-EEG data

Kenneth Borup; Preben Kidmose; Huy Phan; Kaare Mikkelsen

doi:10.1016/j.bspc.2022.104496

Automatic sleep scoring using patient-specific ensemble models and knowledge distillation for ear-EEG data

Kenneth Borup, Preben Kidmose, Huy Phan, Kaare Mikkelsen

Publikation: Bidrag til tidsskrift/Konferencebidrag i tidsskrift /Bidrag til avis › Tidsskriftartikel › Forskning › peer review

9 Citationer (Scopus)

7 Downloads (Pure)

Abstract

Human sleep can be described as a series of transitions between distinct states. This makes automatic sleep analysis (scoring) suitable for an automatic implementation using machine learning. However, the task becomes harder when data is sampled using more light-weight or mobile equipment, often chosen due to greater comfort for the patient. In this study we investigate the improvement in sleep scoring when multiple state-of-the-art neural networks are joined into an ensemble, and subsequently distilled into a single model of identical network architecture, but with improved predictive performance. In this study we investigate ensembles of up to 10 networks, and show that, on the same data, ensembles of neural networks perform better than each single subject model (improvement: 2.4 and that this improvement can be transferred back into a single network using a combination of patient specific data and knowledge distillation. The study demonstrates both a way to further improve automatic sleep scoring from mobile devices, which in itself is interesting, but also highlights the great potential of the vast amounts of unlabeled personal data which will become available from personal recording devices.

Originalsprog	Engelsk
Artikelnummer	104496
Tidsskrift	Biomedical Signal Processing and Control
Vol/bind	81
Antal sider	9
ISSN	1746-8094
DOI	https://doi.org/10.1016/j.bspc.2022.104496
Status	Udgivet - mar. 2023

Adgang til dokumentet

10.1016/j.bspc.2022.104496Licens: CC BY

1-s2.0-S1746809422009508-mainForlagets udgivne version, 1,43 MBLicens: CC BY

Biblioteksadgang?

Tjek tilgængelighed hos det Kgl. Bibliotek

Andre filer og links

Link to publication in Scopus

Citationsformater

@article{0014228be3e74e2a8ec2f0419a38c3e5,

title = "Automatic sleep scoring using patient-specific ensemble models and knowledge distillation for ear-EEG data",

abstract = "Human sleep can be described as a series of transitions between distinct states. This makes automatic sleep analysis (scoring) suitable for an automatic implementation using machine learning. However, the task becomes harder when data is sampled using more light-weight or mobile equipment, often chosen due to greater comfort for the patient. In this study we investigate the improvement in sleep scoring when multiple state-of-the-art neural networks are joined into an ensemble, and subsequently distilled into a single model of identical network architecture, but with improved predictive performance. In this study we investigate ensembles of up to 10 networks, and show that, on the same data, ensembles of neural networks perform better than each single subject model (improvement: 2.4 and that this improvement can be transferred back into a single network using a combination of patient specific data and knowledge distillation. The study demonstrates both a way to further improve automatic sleep scoring from mobile devices, which in itself is interesting, but also highlights the great potential of the vast amounts of unlabeled personal data which will become available from personal recording devices.",

keywords = "Automatic sleep scoring, Ensemble models, Knowledge distillation, Light-weight sleep scoring, Personalized models, Semi-supervised learning",

author = "Kenneth Borup and Preben Kidmose and Huy Phan and Kaare Mikkelsen",

year = "2023",

month = mar,

doi = "10.1016/j.bspc.2022.104496",

language = "English",

volume = "81",

journal = "Biomedical Signal Processing and Control",

issn = "1746-8094",

publisher = "Elsevier Ltd",

}

TY - JOUR

T1 - Automatic sleep scoring using patient-specific ensemble models and knowledge distillation for ear-EEG data

AU - Borup, Kenneth

AU - Kidmose, Preben

AU - Phan, Huy

AU - Mikkelsen, Kaare

PY - 2023/3

Y1 - 2023/3

N2 - Human sleep can be described as a series of transitions between distinct states. This makes automatic sleep analysis (scoring) suitable for an automatic implementation using machine learning. However, the task becomes harder when data is sampled using more light-weight or mobile equipment, often chosen due to greater comfort for the patient. In this study we investigate the improvement in sleep scoring when multiple state-of-the-art neural networks are joined into an ensemble, and subsequently distilled into a single model of identical network architecture, but with improved predictive performance. In this study we investigate ensembles of up to 10 networks, and show that, on the same data, ensembles of neural networks perform better than each single subject model (improvement: 2.4 and that this improvement can be transferred back into a single network using a combination of patient specific data and knowledge distillation. The study demonstrates both a way to further improve automatic sleep scoring from mobile devices, which in itself is interesting, but also highlights the great potential of the vast amounts of unlabeled personal data which will become available from personal recording devices.

AB - Human sleep can be described as a series of transitions between distinct states. This makes automatic sleep analysis (scoring) suitable for an automatic implementation using machine learning. However, the task becomes harder when data is sampled using more light-weight or mobile equipment, often chosen due to greater comfort for the patient. In this study we investigate the improvement in sleep scoring when multiple state-of-the-art neural networks are joined into an ensemble, and subsequently distilled into a single model of identical network architecture, but with improved predictive performance. In this study we investigate ensembles of up to 10 networks, and show that, on the same data, ensembles of neural networks perform better than each single subject model (improvement: 2.4 and that this improvement can be transferred back into a single network using a combination of patient specific data and knowledge distillation. The study demonstrates both a way to further improve automatic sleep scoring from mobile devices, which in itself is interesting, but also highlights the great potential of the vast amounts of unlabeled personal data which will become available from personal recording devices.

KW - Automatic sleep scoring

KW - Ensemble models

KW - Knowledge distillation

KW - Light-weight sleep scoring

KW - Personalized models

KW - Semi-supervised learning

UR - http://www.scopus.com/inward/record.url?scp=85144415184&partnerID=8YFLogxK

U2 - 10.1016/j.bspc.2022.104496

DO - 10.1016/j.bspc.2022.104496

M3 - Journal article

SN - 1746-8094

VL - 81

JO - Biomedical Signal Processing and Control

JF - Biomedical Signal Processing and Control

M1 - 104496

ER -