Abstract
In management research, fixed alpha levels in statistical testing are ubiquitous. However, in highly powered studies, they can lead to Lindley’s paradox, a situation where the null hypothesis is rejected despite evidence in the test actually supporting it. We propose a sample-size-dependent alpha level that combines the benefits of both frequentist and Bayesian statistics, enabling strict hypothesis testing with known error rates while also quantifying the evidence for a hypothesis. We offer actionable guidelines of how to implement the sample-size-dependent alpha in practice and provide an R-package and web app to implement our method for regression models. By using this approach, researchers can avoid mindless defaults and instead justify alpha as a function of sample size, thus improving the reliability of statistical analysis in management research.
Original language | English |
---|---|
Journal | Strategic Organization |
Volume | 22 |
Issue | 3 |
Pages (from-to) | 550-581 |
Number of pages | 32 |
ISSN | 1476-1270 |
DOIs | |
Publication status | Published - Aug 2024 |
Keywords
- Bayes factor
- Bayesian-frequentist compromise
- alpha level
- hypothesis testing
- significance testing