Epistemic consequences of unfair tools

TY - JOUR

T1 - Epistemic consequences of unfair tools

AU - Lassen, Ida Marie S.

AU - Kristensen-McLachlan, Ross Deans

AU - Almasi, Mina

AU - Enevoldsen, Kenneth

AU - Nielbo, Kristoffer Laigaard

PY - 2024/4

Y1 - 2024/4

N2 - This article examines the epistemic consequences of unfair technologies used in digital humanities (DH). We connect bias analysis informed by the field of algorithmic fairness with perspectives on knowledge production in DH. We examine the fairness of Danish Named Entity Recognition tools through an innovative experimental method involving data augmentation and evaluate the performance disparities based on two metrics of algorithmic fairness: calibration within groups; and balance for the positive class. Our results show that only two of the ten tested models comply with the fairness criteria. From an intersectional perspective, we shed light on how unequal performance across groups can lead to the exclusion and marginalization of certain social groups, leading to voices and experiences being disregarded and silenced. We propose incorporating algorithmic fairness in the selection of tools in DH to help alleviate the risk of perpetuating silence and move towards fairer and more inclusive research.

AB - This article examines the epistemic consequences of unfair technologies used in digital humanities (DH). We connect bias analysis informed by the field of algorithmic fairness with perspectives on knowledge production in DH. We examine the fairness of Danish Named Entity Recognition tools through an innovative experimental method involving data augmentation and evaluate the performance disparities based on two metrics of algorithmic fairness: calibration within groups; and balance for the positive class. Our results show that only two of the ten tested models comply with the fairness criteria. From an intersectional perspective, we shed light on how unequal performance across groups can lead to the exclusion and marginalization of certain social groups, leading to voices and experiences being disregarded and silenced. We propose incorporating algorithmic fairness in the selection of tools in DH to help alleviate the risk of perpetuating silence and move towards fairer and more inclusive research.

KW - algorithmic fairness

KW - bias

KW - intersectionality

KW - knowledge production

KW - named entity recognition

KW - natural language processing

U2 - 10.1093/llc/fqad091

DO - 10.1093/llc/fqad091

M3 - Journal article

SN - 2055-7671

VL - 39

SP - 198

EP - 214

JO - Digital Scholarship in the Humanities

JF - Digital Scholarship in the Humanities

IS - 1

ER -