Dansk Center for Forskningsanalyse

Null hypothesis significance tests. A mix-up of two different theories: the basis for widespread confusion and numerous misinterpretations

Publikation: Bidrag til tidsskrift/Konferencebidrag i tidsskrift /Bidrag til avisTidsskriftartikelForskningpeer review

Standard

Null hypothesis significance tests. A mix-up of two different theories : the basis for widespread confusion and numerous misinterpretations. / Schneider, Jesper Wiborg.

I: Scientometrics, Bind 102, Nr. 1, 2015, s. 411-432.

Publikation: Bidrag til tidsskrift/Konferencebidrag i tidsskrift /Bidrag til avisTidsskriftartikelForskningpeer review

Harvard

APA

CBE

MLA

Vancouver

Author

Bibtex

@article{21daef47808d4e06828006fa044d2643,
title = "Null hypothesis significance tests. A mix-up of two different theories: the basis for widespread confusion and numerous misinterpretations",
abstract = "Null hypothesis statistical significance tests (NHST) are widely used in quantitative research in the empirical sciences including scientometrics. Nevertheless, since their introduction nearly a century ago significance tests have been controversial. Many researchers are not aware of the numerous criticisms raised against NHST. As practiced, NHST has been characterized as a {\textquoteleft}null ritual{\textquoteright} that is overused and too often misapplied and misinterpreted. NHST is in fact a patchwork of two fundamentally different classical statistical testing models, often blended with some wishful quasi-Bayesian interpretations. This is undoubtedly a major reason why NHST is very often misunderstood. But NHST also has intrinsic logical problems and the epistemic range of the information provided by such tests is much more limited than most researchers recognize. In this article we introduce to the scientometric community the theoretical origins of NHST, which is mostly absent from standard statistical textbooks, and we discuss some of the most prevalent problems relating to the practice of NHST and trace these problems back to the mix-up of the two different theoretical origins. Finally, we illustrate some of the misunderstandings with examples from the scientometric literature and bring forward some modest recommendations for a more sound practice in quantitative data analysis.",
keywords = "statistical significance tests, scientometrics",
author = "Schneider, {Jesper Wiborg}",
note = "2016 ISSI Award for Best Scientometric paper in any journal in 2015",
year = "2015",
doi = "10.1007/s11192-014-1251-5",
language = "English",
volume = "102",
pages = "411--432",
journal = "Scientometrics",
issn = "0138-9130",
publisher = "Akademiai Kiado Rt.",
number = "1",

}

RIS

TY - JOUR

T1 - Null hypothesis significance tests. A mix-up of two different theories

T2 - the basis for widespread confusion and numerous misinterpretations

AU - Schneider, Jesper Wiborg

N1 - 2016 ISSI Award for Best Scientometric paper in any journal in 2015

PY - 2015

Y1 - 2015

N2 - Null hypothesis statistical significance tests (NHST) are widely used in quantitative research in the empirical sciences including scientometrics. Nevertheless, since their introduction nearly a century ago significance tests have been controversial. Many researchers are not aware of the numerous criticisms raised against NHST. As practiced, NHST has been characterized as a ‘null ritual’ that is overused and too often misapplied and misinterpreted. NHST is in fact a patchwork of two fundamentally different classical statistical testing models, often blended with some wishful quasi-Bayesian interpretations. This is undoubtedly a major reason why NHST is very often misunderstood. But NHST also has intrinsic logical problems and the epistemic range of the information provided by such tests is much more limited than most researchers recognize. In this article we introduce to the scientometric community the theoretical origins of NHST, which is mostly absent from standard statistical textbooks, and we discuss some of the most prevalent problems relating to the practice of NHST and trace these problems back to the mix-up of the two different theoretical origins. Finally, we illustrate some of the misunderstandings with examples from the scientometric literature and bring forward some modest recommendations for a more sound practice in quantitative data analysis.

AB - Null hypothesis statistical significance tests (NHST) are widely used in quantitative research in the empirical sciences including scientometrics. Nevertheless, since their introduction nearly a century ago significance tests have been controversial. Many researchers are not aware of the numerous criticisms raised against NHST. As practiced, NHST has been characterized as a ‘null ritual’ that is overused and too often misapplied and misinterpreted. NHST is in fact a patchwork of two fundamentally different classical statistical testing models, often blended with some wishful quasi-Bayesian interpretations. This is undoubtedly a major reason why NHST is very often misunderstood. But NHST also has intrinsic logical problems and the epistemic range of the information provided by such tests is much more limited than most researchers recognize. In this article we introduce to the scientometric community the theoretical origins of NHST, which is mostly absent from standard statistical textbooks, and we discuss some of the most prevalent problems relating to the practice of NHST and trace these problems back to the mix-up of the two different theoretical origins. Finally, we illustrate some of the misunderstandings with examples from the scientometric literature and bring forward some modest recommendations for a more sound practice in quantitative data analysis.

KW - statistical significance tests

KW - scientometrics

U2 - 10.1007/s11192-014-1251-5

DO - 10.1007/s11192-014-1251-5

M3 - Journal article

VL - 102

SP - 411

EP - 432

JO - Scientometrics

JF - Scientometrics

SN - 0138-9130

IS - 1

ER -