Department of Political Science

Question Order Bias Revisited: A Split‐Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review

Standard

Question Order Bias Revisited : A Split‐Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users. / Thau, Mads; Mikkelsen, Maria Falk; Hjortskov, Morten; Pedersen, Mogens Jin.

In: Public Administration, Vol. 99, No. 1, 03.2021, p. 189-204.

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review

Harvard

Thau, M, Mikkelsen, MF, Hjortskov, M & Pedersen, MJ 2021, 'Question Order Bias Revisited: A Split‐Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users', Public Administration, vol. 99, no. 1, pp. 189-204. https://doi.org/10.1111/padm.12688

APA

CBE

MLA

Vancouver

Author

Thau, Mads ; Mikkelsen, Maria Falk ; Hjortskov, Morten ; Pedersen, Mogens Jin. / Question Order Bias Revisited : A Split‐Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users. In: Public Administration. 2021 ; Vol. 99, No. 1. pp. 189-204.

Bibtex

@article{2cacf0723efe415b92869005db507e9c,
title = "Question Order Bias Revisited: A Split‐Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users",
abstract = "Public decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.",
keywords = "Citizen Satisfaction, Experiment, Question order",
author = "Mads Thau and Mikkelsen, {Maria Falk} and Morten Hjortskov and Pedersen, {Mogens Jin}",
year = "2021",
month = mar,
doi = "10.1111/padm.12688",
language = "English",
volume = "99",
pages = "189--204",
journal = "Public Administration",
issn = "0033-3298",
publisher = "Wiley-Blackwell Publishing Ltd.",
number = "1",

}

RIS

TY - JOUR

T1 - Question Order Bias Revisited

T2 - A Split‐Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users

AU - Thau, Mads

AU - Mikkelsen, Maria Falk

AU - Hjortskov, Morten

AU - Pedersen, Mogens Jin

PY - 2021/3

Y1 - 2021/3

N2 - Public decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.

AB - Public decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.

KW - Citizen Satisfaction

KW - Experiment

KW - Question order

UR - http://www.scopus.com/inward/record.url?scp=85088948138&partnerID=8YFLogxK

U2 - 10.1111/padm.12688

DO - 10.1111/padm.12688

M3 - Journal article

VL - 99

SP - 189

EP - 204

JO - Public Administration

JF - Public Administration

SN - 0033-3298

IS - 1

ER -