Measuring Literary Quality. Proxies and Perspectives

Pascale Feldkamp; Yuri Bizzoni; Mads Rosendahl Thomsen; Kristoffer L. Nielbo

doi:10.26083/tuprints-00027391

Measuring Literary Quality. Proxies and Perspectives

Research output: Working paper/Preprint › Preprint

Abstract

Computational studies of literature have adopted approaches from statistics and social sciences to perform large scale studies of fiction, and recent work has sought to approximate the success of literary texts using some proxy for literary quality, such as collections of human judgments, sales-numbers or lists indicating canonicity. However, most quantitative studies of literary quality use one such measure as a golden standard of literary judgement without fully reflecting on what it represents. Conclusions drawn from these studies are nonetheless bound to mirror a particular conception of literary quality associated with the chosen metric. To address this issue, we provide a discussion of the interrelation of various “proxies of literary quality” within a corpus of novels published in the US in the late 19th and 20th century, performing correlations and comparisons across 14 different proxies. We start with a heuristic distinction between expert-based literary judgments, such as those represented by college syllabi and literary anthologies, and crowd-based judgments, such as GoodReads’ ratings, and explore the differences between these and other proxies that fall in-between, such as library holding numbers, prestigious literary prizes, and classics book series. Our findings suggest that works favored in expert-based judgments tend to score lower on GoodReads, while those longlisted for awards tend to score higher and enjoy greater circulation in libraries. Generally, two main kinds of “quality perception” emerge as we map the literary judgment landscape: one associated with canonical literature, and one with more popular literature, which may indicate that judgements of canonicity or literariness are not equal to popularity among readers. Additionally, our study suggests that prestige in genre-literature, as represented by main genre-fiction awards such as the Hugo or World Fantasy Award, constitute distinct proxies on their own, though more closely aligned to popular than canonical proxies.

Original language	English
Publisher	Universitäts- und Landesbibliothek Darmstadt
DOIs	https://doi.org/10.26083/tuprints-00027391
Publication status	Published - 1 May 2024

Access to Document

10.26083/tuprints-00027391Licence: CC BY

https://tuprints.ulb.tu-darmstadt.de/27391/

Library access?

Check availability with the Royal Danish Library

Cite this

@misc{7acec912b3914693b53ff33c36393949,

title = "Measuring Literary Quality. Proxies and Perspectives",

abstract = "Computational studies of literature have adopted approaches from statistics and social sciences to perform large scale studies of fiction, and recent work has sought to approximate the success of literary texts using some proxy for literary quality, such as collections of human judgments, sales-numbers or lists indicating canonicity. However, most quantitative studies of literary quality use one such measure as a golden standard of literary judgement without fully reflecting on what it represents. Conclusions drawn from these studies are nonetheless bound to mirror a particular conception of literary quality associated with the chosen metric. To address this issue, we provide a discussion of the interrelation of various “proxies of literary quality” within a corpus of novels published in the US in the late 19th and 20th century, performing correlations and comparisons across 14 different proxies. We start with a heuristic distinction between expert-based literary judgments, such as those represented by college syllabi and literary anthologies, and crowd-based judgments, such as GoodReads{\textquoteright} ratings, and explore the differences between these and other proxies that fall in-between, such as library holding numbers, prestigious literary prizes, and classics book series. Our findings suggest that works favored in expert-based judgments tend to score lower on GoodReads, while those longlisted for awards tend to score higher and enjoy greater circulation in libraries. Generally, two main kinds of “quality perception” emerge as we map the literary judgment landscape: one associated with canonical literature, and one with more popular literature, which may indicate that judgements of canonicity or literariness are not equal to popularity among readers. Additionally, our study suggests that prestige in genre-literature, as represented by main genre-fiction awards such as the Hugo or World Fantasy Award, constitute distinct proxies on their own, though more closely aligned to popular than canonical proxies.",

author = "Pascale Feldkamp and Yuri Bizzoni and Thomsen, \{Mads Rosendahl\} and Nielbo, \{Kristoffer L.\}",

note = "Issue: 1 Volume: 3",

year = "2024",

month = may,

day = "1",

doi = "10.26083/tuprints-00027391",

language = "English",

publisher = "Universit{\"a}ts- und Landesbibliothek Darmstadt",

address = "Germany",

type = "WorkingPaper",

institution = "Universit{\"a}ts- und Landesbibliothek Darmstadt",

}

TY - UNPB

T1 - Measuring Literary Quality. Proxies and Perspectives

AU - Feldkamp, Pascale

AU - Bizzoni, Yuri

AU - Thomsen, Mads Rosendahl

AU - Nielbo, Kristoffer L.

N1 - Issue: 1 Volume: 3

PY - 2024/5/1

Y1 - 2024/5/1

N2 - Computational studies of literature have adopted approaches from statistics and social sciences to perform large scale studies of fiction, and recent work has sought to approximate the success of literary texts using some proxy for literary quality, such as collections of human judgments, sales-numbers or lists indicating canonicity. However, most quantitative studies of literary quality use one such measure as a golden standard of literary judgement without fully reflecting on what it represents. Conclusions drawn from these studies are nonetheless bound to mirror a particular conception of literary quality associated with the chosen metric. To address this issue, we provide a discussion of the interrelation of various “proxies of literary quality” within a corpus of novels published in the US in the late 19th and 20th century, performing correlations and comparisons across 14 different proxies. We start with a heuristic distinction between expert-based literary judgments, such as those represented by college syllabi and literary anthologies, and crowd-based judgments, such as GoodReads’ ratings, and explore the differences between these and other proxies that fall in-between, such as library holding numbers, prestigious literary prizes, and classics book series. Our findings suggest that works favored in expert-based judgments tend to score lower on GoodReads, while those longlisted for awards tend to score higher and enjoy greater circulation in libraries. Generally, two main kinds of “quality perception” emerge as we map the literary judgment landscape: one associated with canonical literature, and one with more popular literature, which may indicate that judgements of canonicity or literariness are not equal to popularity among readers. Additionally, our study suggests that prestige in genre-literature, as represented by main genre-fiction awards such as the Hugo or World Fantasy Award, constitute distinct proxies on their own, though more closely aligned to popular than canonical proxies.

AB - Computational studies of literature have adopted approaches from statistics and social sciences to perform large scale studies of fiction, and recent work has sought to approximate the success of literary texts using some proxy for literary quality, such as collections of human judgments, sales-numbers or lists indicating canonicity. However, most quantitative studies of literary quality use one such measure as a golden standard of literary judgement without fully reflecting on what it represents. Conclusions drawn from these studies are nonetheless bound to mirror a particular conception of literary quality associated with the chosen metric. To address this issue, we provide a discussion of the interrelation of various “proxies of literary quality” within a corpus of novels published in the US in the late 19th and 20th century, performing correlations and comparisons across 14 different proxies. We start with a heuristic distinction between expert-based literary judgments, such as those represented by college syllabi and literary anthologies, and crowd-based judgments, such as GoodReads’ ratings, and explore the differences between these and other proxies that fall in-between, such as library holding numbers, prestigious literary prizes, and classics book series. Our findings suggest that works favored in expert-based judgments tend to score lower on GoodReads, while those longlisted for awards tend to score higher and enjoy greater circulation in libraries. Generally, two main kinds of “quality perception” emerge as we map the literary judgment landscape: one associated with canonical literature, and one with more popular literature, which may indicate that judgements of canonicity or literariness are not equal to popularity among readers. Additionally, our study suggests that prestige in genre-literature, as represented by main genre-fiction awards such as the Hugo or World Fantasy Award, constitute distinct proxies on their own, though more closely aligned to popular than canonical proxies.

U2 - 10.26083/tuprints-00027391

DO - 10.26083/tuprints-00027391

M3 - Preprint

BT - Measuring Literary Quality. Proxies and Perspectives

PB - Universitäts- und Landesbibliothek Darmstadt

ER -