Aarhus Universitets segl

A machine learning perspective on the emotional content of Parkinsonian speech

Publikation: Bidrag til tidsskrift/Konferencebidrag i tidsskrift /Bidrag til avisTidsskriftartikelForskningpeer review

Standard

A machine learning perspective on the emotional content of Parkinsonian speech. / Sechidis, Konstantinos; Fusaroli, Riccardo; Orozco-Arroyave, Juan Rafael et al.

I: Artificial Intelligence in Medicine, Bind 115, 102061, 05.2021.

Publikation: Bidrag til tidsskrift/Konferencebidrag i tidsskrift /Bidrag til avisTidsskriftartikelForskningpeer review

Harvard

Sechidis, K, Fusaroli, R, Orozco-Arroyave, JR, Wolf, D & Zhang, YP 2021, 'A machine learning perspective on the emotional content of Parkinsonian speech', Artificial Intelligence in Medicine, bind 115, 102061. https://doi.org/10.1016/j.artmed.2021.102061

APA

Sechidis, K., Fusaroli, R., Orozco-Arroyave, J. R., Wolf, D., & Zhang, Y. P. (2021). A machine learning perspective on the emotional content of Parkinsonian speech. Artificial Intelligence in Medicine, 115, [102061]. https://doi.org/10.1016/j.artmed.2021.102061

CBE

Sechidis K, Fusaroli R, Orozco-Arroyave JR, Wolf D, Zhang YP. 2021. A machine learning perspective on the emotional content of Parkinsonian speech. Artificial Intelligence in Medicine. 115:Article 102061. https://doi.org/10.1016/j.artmed.2021.102061

MLA

Vancouver

Sechidis K, Fusaroli R, Orozco-Arroyave JR, Wolf D, Zhang YP. A machine learning perspective on the emotional content of Parkinsonian speech. Artificial Intelligence in Medicine. 2021 maj;115:102061. doi: 10.1016/j.artmed.2021.102061

Author

Sechidis, Konstantinos ; Fusaroli, Riccardo ; Orozco-Arroyave, Juan Rafael et al. / A machine learning perspective on the emotional content of Parkinsonian speech. I: Artificial Intelligence in Medicine. 2021 ; Bind 115.

Bibtex

@article{00e64706d59348269a77454e7cecc215,
title = "A machine learning perspective on the emotional content of Parkinsonian speech",
abstract = "Patients with Parkinson's disease (PD) have distinctive voice patterns, often perceived as expressing sad emotion. While this characteristic of Parkinsonian speech has been supported through the perspective of listeners, where both PD and healthy control (HC) subjects repeat the same speaking tasks, it has never been explored through a machine learning modelling approach. Our work provides an objective evaluation of this characteristic of the PD speech, by building a transfer learning system to assess how the PD pathology affects the sadness perception. To do so we introduce a Mixture-of-Experts (MoE) architecture for speech emotion recognition designed to be transferable across datasets. Firstly, by relying on publicly available emotional speech corpora, we train the MoE model and then we use it to quantify perceived sadness in never seen before PD and matched HC speech recordings. To build our models (experts), we extracted spectral features of the voicing parts of speech and we trained a gradient boosting decision trees model in each corpus to predict happiness vs. sadness. MoE predictions are created by weighting each expert's prediction according to the distance between the new sample and the expert-specific training samples. The MoE approach systematically infers more negative emotional characteristics in PD speech than in HC. Crucially, these judgments are related to the disease severity and the severity of speech impairment in the PD patients: the more impairment, the more likely the speech is to be judged as sad. Our findings pave the way towards a better understanding of the characteristics of PD speech and show how publicly available datasets can be used to train models that provide interesting insights on clinical data.",
keywords = "Machine learning, Mixture-of-experts, Parkinson's disease, Speech emotion recognition",
author = "Konstantinos Sechidis and Riccardo Fusaroli and Orozco-Arroyave, {Juan Rafael} and Detlef Wolf and Zhang, {Yan Ping}",
note = "Publisher Copyright: {\textcopyright} 2021 The Authors Copyright: Copyright 2021 Elsevier B.V., All rights reserved.",
year = "2021",
month = may,
doi = "10.1016/j.artmed.2021.102061",
language = "English",
volume = "115",
journal = "Artificial Intelligence in Medicine",
issn = "0933-3657",
publisher = "Elsevier BV",

}

RIS

TY - JOUR

T1 - A machine learning perspective on the emotional content of Parkinsonian speech

AU - Sechidis, Konstantinos

AU - Fusaroli, Riccardo

AU - Orozco-Arroyave, Juan Rafael

AU - Wolf, Detlef

AU - Zhang, Yan Ping

N1 - Publisher Copyright: © 2021 The Authors Copyright: Copyright 2021 Elsevier B.V., All rights reserved.

PY - 2021/5

Y1 - 2021/5

N2 - Patients with Parkinson's disease (PD) have distinctive voice patterns, often perceived as expressing sad emotion. While this characteristic of Parkinsonian speech has been supported through the perspective of listeners, where both PD and healthy control (HC) subjects repeat the same speaking tasks, it has never been explored through a machine learning modelling approach. Our work provides an objective evaluation of this characteristic of the PD speech, by building a transfer learning system to assess how the PD pathology affects the sadness perception. To do so we introduce a Mixture-of-Experts (MoE) architecture for speech emotion recognition designed to be transferable across datasets. Firstly, by relying on publicly available emotional speech corpora, we train the MoE model and then we use it to quantify perceived sadness in never seen before PD and matched HC speech recordings. To build our models (experts), we extracted spectral features of the voicing parts of speech and we trained a gradient boosting decision trees model in each corpus to predict happiness vs. sadness. MoE predictions are created by weighting each expert's prediction according to the distance between the new sample and the expert-specific training samples. The MoE approach systematically infers more negative emotional characteristics in PD speech than in HC. Crucially, these judgments are related to the disease severity and the severity of speech impairment in the PD patients: the more impairment, the more likely the speech is to be judged as sad. Our findings pave the way towards a better understanding of the characteristics of PD speech and show how publicly available datasets can be used to train models that provide interesting insights on clinical data.

AB - Patients with Parkinson's disease (PD) have distinctive voice patterns, often perceived as expressing sad emotion. While this characteristic of Parkinsonian speech has been supported through the perspective of listeners, where both PD and healthy control (HC) subjects repeat the same speaking tasks, it has never been explored through a machine learning modelling approach. Our work provides an objective evaluation of this characteristic of the PD speech, by building a transfer learning system to assess how the PD pathology affects the sadness perception. To do so we introduce a Mixture-of-Experts (MoE) architecture for speech emotion recognition designed to be transferable across datasets. Firstly, by relying on publicly available emotional speech corpora, we train the MoE model and then we use it to quantify perceived sadness in never seen before PD and matched HC speech recordings. To build our models (experts), we extracted spectral features of the voicing parts of speech and we trained a gradient boosting decision trees model in each corpus to predict happiness vs. sadness. MoE predictions are created by weighting each expert's prediction according to the distance between the new sample and the expert-specific training samples. The MoE approach systematically infers more negative emotional characteristics in PD speech than in HC. Crucially, these judgments are related to the disease severity and the severity of speech impairment in the PD patients: the more impairment, the more likely the speech is to be judged as sad. Our findings pave the way towards a better understanding of the characteristics of PD speech and show how publicly available datasets can be used to train models that provide interesting insights on clinical data.

KW - Machine learning

KW - Mixture-of-experts

KW - Parkinson's disease

KW - Speech emotion recognition

UR - http://www.scopus.com/inward/record.url?scp=85104321177&partnerID=8YFLogxK

U2 - 10.1016/j.artmed.2021.102061

DO - 10.1016/j.artmed.2021.102061

M3 - Journal article

C2 - 34001321

AN - SCOPUS:85104321177

VL - 115

JO - Artificial Intelligence in Medicine

JF - Artificial Intelligence in Medicine

SN - 0933-3657

M1 - 102061

ER -