The Danish Centre for Studies in Research and Research Policy

Jesper Wiborg Schneider

Are larger effect sizes in experimental studies good predictors of higher citation rates? A Bayesian examination

Research output: Contribution to book/anthology/report/proceedingArticle in proceedingsResearchpeer-review

Documents

Links

Effect sizes are perhaps the most important quantitative information in statistical
inferential studies. Recently, the hypothesis that rational citation behaviour in general ought to give credit to studies that successfully apply a treatment and detect greater effects, resulting in such studies being cited more frequently among comparable studies. Hence, it is predicted that larger effect sizes increases study relative citation rates. Two recent studies in biology provide contradictory results on this hypothesis. The present study investigates the same hypothesis but in different research areas and with a more credible model selection procedure.
Using meta-analyses, we identify comparable individual experimental studies (n=259) from five different research specialties. Effect sizes are compared to the citation rates of the individual studies and impact factors for the journals where the studies are published. Contrary to the previous findings, and in fact most studies in scientometrics, we examine the hypothesis with a Bayesian model selection procedure. This is advantageous, as we thereby are able to quantify the statistical evidence for both hypotheses, H0 and H1. This is not possible in classical statistical inference, though the implicit inferential decision
made by most researchers when they fail to reject H0 is to accept it. This is a flawed logic. Given uniform priors for the two hypotheses, the result from the present data set is posterior odds of 13/4 to 1 in favor of the null models examined. Consequently, the study give positive evidence to the claim made by Lortie et al. (forthcoming) that effect sizes do not predict citation rates and are as such poor proxies for the quantitative merit of a given experimental treatment.
Original languageEnglish
Title of host publicationProceedings of the 14th ISSI Conference : 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, 15th to 20th July 2013
EditorsJuan Gorraiz, Edgar Schiebel, Christian Gumpenberger, Marianne Hörlesberger, Henk Moed
Volume1
Publication year2013
Pages152-166
ISBN (print)978-3-200-03135-7
Publication statusPublished - 2013

    Research areas

  • scientometrics, Bayesian statistic, meta analysis

See relations at Aarhus University Citationformats

Download statistics

No data available

ID: 70917203