'Super-Unsupervised' Classification for Labelling Text: Online Political Hostility as an Illustration

Stig Hebbelstrup Rye Rasmussen; Alexander Bor; Mathias Osmundsen; Michael Bang Petersen

doi:10.1017/S0007123423000042

'Super-Unsupervised' Classification for Labelling Text: Online Political Hostility as an Illustration

Stig Hebbelstrup Rye Rasmussen^*, Alexander Bor, Mathias Osmundsen, Michael Bang Petersen

^*Corresponding author for this work

Department of Political Science

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaper › Journal article › Research › peer-review

2 Citations (Scopus)

Abstract

We live in a world of text. Yet the sheer magnitude of social media data, coupled with a need to measure complex psychological constructs, has made this important source of data difficult to use. Researchers often engage in costly hand coding of thousands of texts using supervised techniques or rely on unsupervised techniques where the measurement of predefined constructs is difficult. We propose a novel approach that we call 'super-unsupervised' learning and demonstrate its usefulness by measuring the psychologically complex construct of online political hostility based on a large corpus of tweets. This approach accomplishes the feat by combining the best features of supervised and unsupervised learning techniques: measurements of complex psychological constructs without a single labelled data source. We first outline the approach before conducting a diverse series of tests that include: (i) face validity, (ii) convergent and discriminant validity, (iii) criterion validity, (iv) external validity, and (v) ecological validity.

Original language	English
Journal	British Journal of Political Science
Volume	54
Issue	1
Pages (from-to)	179-200
Number of pages	22
ISSN	0007-1234
DOIs	https://doi.org/10.1017/S0007123423000042
Publication status	Published - 24 Jan 2024

Keywords

natural language processing
social media
supervised learning
unsupervised learning

Access to Document

10.1017/S0007123423000042Licence: CC BY

Cite this

@article{74d265b7c5a34433975719ae49b890f0,

title = "'Super-Unsupervised' Classification for Labelling Text: Online Political Hostility as an Illustration",

abstract = "We live in a world of text. Yet the sheer magnitude of social media data, coupled with a need to measure complex psychological constructs, has made this important source of data difficult to use. Researchers often engage in costly hand coding of thousands of texts using supervised techniques or rely on unsupervised techniques where the measurement of predefined constructs is difficult. We propose a novel approach that we call 'super-unsupervised' learning and demonstrate its usefulness by measuring the psychologically complex construct of online political hostility based on a large corpus of tweets. This approach accomplishes the feat by combining the best features of supervised and unsupervised learning techniques: measurements of complex psychological constructs without a single labelled data source. We first outline the approach before conducting a diverse series of tests that include: (i) face validity, (ii) convergent and discriminant validity, (iii) criterion validity, (iv) external validity, and (v) ecological validity.",

keywords = "natural language processing, social media, supervised learning, unsupervised learning",

author = "{Hebbelstrup Rye Rasmussen}, Stig and Alexander Bor and Mathias Osmundsen and Petersen, {Michael Bang}",

note = "Publisher Copyright: Copyright {\textcopyright} The Author(s), 2023. Published by Cambridge University Press.",

year = "2024",

month = jan,

day = "24",

doi = "10.1017/S0007123423000042",

language = "English",

volume = "54",

pages = "179--200",

journal = "British Journal of Political Science",

issn = "0007-1234",

publisher = "Cambridge University Press",

number = "1",

}

'Super-Unsupervised' Classification for Labelling Text: Online Political Hostility as an Illustration. / Hebbelstrup Rye Rasmussen, Stig; Bor, Alexander; Osmundsen, Mathias et al.
In: British Journal of Political Science, Vol. 54, No. 1, 24.01.2024, p. 179-200.

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaper › Journal article › Research › peer-review

TY - JOUR

T1 - 'Super-Unsupervised' Classification for Labelling Text

T2 - Online Political Hostility as an Illustration

AU - Hebbelstrup Rye Rasmussen, Stig

AU - Bor, Alexander

AU - Osmundsen, Mathias

AU - Petersen, Michael Bang

PY - 2024/1/24

Y1 - 2024/1/24

N2 - We live in a world of text. Yet the sheer magnitude of social media data, coupled with a need to measure complex psychological constructs, has made this important source of data difficult to use. Researchers often engage in costly hand coding of thousands of texts using supervised techniques or rely on unsupervised techniques where the measurement of predefined constructs is difficult. We propose a novel approach that we call 'super-unsupervised' learning and demonstrate its usefulness by measuring the psychologically complex construct of online political hostility based on a large corpus of tweets. This approach accomplishes the feat by combining the best features of supervised and unsupervised learning techniques: measurements of complex psychological constructs without a single labelled data source. We first outline the approach before conducting a diverse series of tests that include: (i) face validity, (ii) convergent and discriminant validity, (iii) criterion validity, (iv) external validity, and (v) ecological validity.

AB - We live in a world of text. Yet the sheer magnitude of social media data, coupled with a need to measure complex psychological constructs, has made this important source of data difficult to use. Researchers often engage in costly hand coding of thousands of texts using supervised techniques or rely on unsupervised techniques where the measurement of predefined constructs is difficult. We propose a novel approach that we call 'super-unsupervised' learning and demonstrate its usefulness by measuring the psychologically complex construct of online political hostility based on a large corpus of tweets. This approach accomplishes the feat by combining the best features of supervised and unsupervised learning techniques: measurements of complex psychological constructs without a single labelled data source. We first outline the approach before conducting a diverse series of tests that include: (i) face validity, (ii) convergent and discriminant validity, (iii) criterion validity, (iv) external validity, and (v) ecological validity.

KW - natural language processing

KW - social media

KW - supervised learning

KW - unsupervised learning

UR - http://www.scopus.com/inward/record.url?scp=85179464907&partnerID=8YFLogxK

U2 - 10.1017/S0007123423000042

DO - 10.1017/S0007123423000042

M3 - Journal article

AN - SCOPUS:85179464907

SN - 0007-1234

VL - 54

SP - 179

EP - 200

JO - British Journal of Political Science

JF - British Journal of Political Science

IS - 1

ER -

'Super-Unsupervised' Classification for Labelling Text: Online Political Hostility as an Illustration

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this