Dansk Center for Forskningsanalyse

Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination

Publikation: Bidrag til bog/antologi/rapport/proceedingKonferencebidrag i proceedingsForskningpeer review

Standard

Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination. / Schneider, Jesper Wiborg; Henriksen, Dorte.

Proceedings of the 14th ISSI Conference: 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013. red. / Juan Gorraiz; Edgar Schiebel; Christian Gumpenberger; Marianne Hörlesberger; Henk Moed. Bind 1 2013. s. 152-166.

Publikation: Bidrag til bog/antologi/rapport/proceedingKonferencebidrag i proceedingsForskningpeer review

Harvard

Schneider, JW & Henriksen, D 2013, Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination. i J Gorraiz, E Schiebel, C Gumpenberger, M Hörlesberger & H Moed (red), Proceedings of the 14th ISSI Conference: 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013. bind 1, s. 152-166. <http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf>

APA

Schneider, J. W., & Henriksen, D. (2013). Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination. I J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (red.), Proceedings of the 14th ISSI Conference: 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013 (Bind 1, s. 152-166) http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf

CBE

Schneider JW, Henriksen D. 2013. Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination. Gorraiz J, Schiebel E, Gumpenberger C, Hörlesberger M, Moed H, red. I Proceedings of the 14th ISSI Conference: 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013. s. 152-166.

MLA

Schneider, Jesper Wiborg og Dorte Henriksen "Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination"., Gorraiz, Juan, Schiebel, Edgar Gumpenberger, Christian Hörlesberger, Marianne Moed, Henk (red.). Proceedings of the 14th ISSI Conference: 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013. 2013, 152-166.

Vancouver

Schneider JW, Henriksen D. Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination. I Gorraiz J, Schiebel E, Gumpenberger C, Hörlesberger M, Moed H, red., Proceedings of the 14th ISSI Conference: 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013. Bind 1. 2013. s. 152-166

Author

Schneider, Jesper Wiborg ; Henriksen, Dorte. / Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination. Proceedings of the 14th ISSI Conference: 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013. red. / Juan Gorraiz ; Edgar Schiebel ; Christian Gumpenberger ; Marianne Hörlesberger ; Henk Moed. Bind 1 2013. s. 152-166

Bibtex

@inproceedings{735a1a0f98d8462c9ec881d6fe95c20e,
title = "Are larger effect sizes in experimental studies good predictors of higher citation rates?: A Bayesian examination",
abstract = "Effect sizes are perhaps the most important quantitative information in statisticalinferential studies. Recently, the hypothesis that rational citation behaviour in general ought to give credit to studies that successfully apply a treatment and detect greater effects, resulting in such studies being cited more frequently among comparable studies. Hence, it is predicted that larger effect sizes increases study relative citation rates. Two recent studies in biology provide contradictory results on this hypothesis. The present study investigates the same hypothesis but in different research areas and with a more credible model selection procedure.Using meta-analyses, we identify comparable individual experimental studies (n=259) from five different research specialties. Effect sizes are compared to the citation rates of the individual studies and impact factors for the journals where the studies are published. Contrary to the previous findings, and in fact most studies in scientometrics, we examine the hypothesis with a Bayesian model selection procedure. This is advantageous, as we thereby are able to quantify the statistical evidence for both hypotheses, H0 and H1. This is not possible in classical statistical inference, though the implicit inferential decisionmade by most researchers when they fail to reject H0 is to accept it. This is a flawed logic. Given uniform priors for the two hypotheses, the result from the present data set is posterior odds of 13/4 to 1 in favor of the null models examined. Consequently, the study give positive evidence to the claim made by Lortie et al. (forthcoming) that effect sizes do not predict citation rates and are as such poor proxies for the quantitative merit of a given experimental treatment.",
keywords = "scientometrics, Bayesian statistic, meta analysis",
author = "Schneider, {Jesper Wiborg} and Dorte Henriksen",
year = "2013",
language = "English",
isbn = "978-3-200-03135-7",
volume = "1",
pages = "152--166",
editor = "Juan Gorraiz and Edgar Schiebel and Christian Gumpenberger and Marianne H{\"o}rlesberger and Henk Moed",
booktitle = "Proceedings of the 14th ISSI Conference",

}

RIS

TY - GEN

T1 - Are larger effect sizes in experimental studies good predictors of higher citation rates?

T2 - A Bayesian examination

AU - Schneider, Jesper Wiborg

AU - Henriksen, Dorte

PY - 2013

Y1 - 2013

N2 - Effect sizes are perhaps the most important quantitative information in statisticalinferential studies. Recently, the hypothesis that rational citation behaviour in general ought to give credit to studies that successfully apply a treatment and detect greater effects, resulting in such studies being cited more frequently among comparable studies. Hence, it is predicted that larger effect sizes increases study relative citation rates. Two recent studies in biology provide contradictory results on this hypothesis. The present study investigates the same hypothesis but in different research areas and with a more credible model selection procedure.Using meta-analyses, we identify comparable individual experimental studies (n=259) from five different research specialties. Effect sizes are compared to the citation rates of the individual studies and impact factors for the journals where the studies are published. Contrary to the previous findings, and in fact most studies in scientometrics, we examine the hypothesis with a Bayesian model selection procedure. This is advantageous, as we thereby are able to quantify the statistical evidence for both hypotheses, H0 and H1. This is not possible in classical statistical inference, though the implicit inferential decisionmade by most researchers when they fail to reject H0 is to accept it. This is a flawed logic. Given uniform priors for the two hypotheses, the result from the present data set is posterior odds of 13/4 to 1 in favor of the null models examined. Consequently, the study give positive evidence to the claim made by Lortie et al. (forthcoming) that effect sizes do not predict citation rates and are as such poor proxies for the quantitative merit of a given experimental treatment.

AB - Effect sizes are perhaps the most important quantitative information in statisticalinferential studies. Recently, the hypothesis that rational citation behaviour in general ought to give credit to studies that successfully apply a treatment and detect greater effects, resulting in such studies being cited more frequently among comparable studies. Hence, it is predicted that larger effect sizes increases study relative citation rates. Two recent studies in biology provide contradictory results on this hypothesis. The present study investigates the same hypothesis but in different research areas and with a more credible model selection procedure.Using meta-analyses, we identify comparable individual experimental studies (n=259) from five different research specialties. Effect sizes are compared to the citation rates of the individual studies and impact factors for the journals where the studies are published. Contrary to the previous findings, and in fact most studies in scientometrics, we examine the hypothesis with a Bayesian model selection procedure. This is advantageous, as we thereby are able to quantify the statistical evidence for both hypotheses, H0 and H1. This is not possible in classical statistical inference, though the implicit inferential decisionmade by most researchers when they fail to reject H0 is to accept it. This is a flawed logic. Given uniform priors for the two hypotheses, the result from the present data set is posterior odds of 13/4 to 1 in favor of the null models examined. Consequently, the study give positive evidence to the claim made by Lortie et al. (forthcoming) that effect sizes do not predict citation rates and are as such poor proxies for the quantitative merit of a given experimental treatment.

KW - scientometrics

KW - Bayesian statistic

KW - meta analysis

M3 - Article in proceedings

SN - 978-3-200-03135-7

VL - 1

SP - 152

EP - 166

BT - Proceedings of the 14th ISSI Conference

A2 - Gorraiz, Juan

A2 - Schiebel, Edgar

A2 - Gumpenberger, Christian

A2 - Hörlesberger, Marianne

A2 - Moed, Henk

ER -