Fluoroscopic image-based behavior analysis can objectively explain subjective expert assessment of wire navigation skill

TY - JOUR

T1 - Fluoroscopic image-based behavior analysis can objectively explain subjective expert assessment of wire navigation skill

AU - Mattioli, Dominik

AU - Thomas, Geb W

AU - Long, Steven A

AU - Rölfing, Jan Hendrik Duedal

AU - Anderson, Donald

PY - 2024/2

Y1 - 2024/2

N2 - Psychomotor skill and decision-making efficiency in surgical wire navigation can be objectively evaluated by analysis of intraoperative fluoroscopic image sequences. Prior work suggests that such image-based behavior analysis of operating room (OR) performance can predict performer experience level (R2 = 0.62) and agree with expert opinion (the current standard) on the quality of a final implant construct (R2 = 0.59). However, it is unclear how objective image-based evaluation compares with expert assessments for entire technical OR performances. This study examines the relationships between three key variables: (1) objective image-based criteria, (2) expert opinions, and (3) performing surgeon experience level. A paired-comparison survey of seven experts done based upon eight OR fluoroscopic wire navigation sequences shows that the experts’ preferences are best explained by objective metrics that reflect psychomotor and decision-making behaviors which are counter-productive to successful implant placement, like image count (R2 = 0.83) and behavior tally (R2 = 0.74). One such behavior, adjustments away from goal, uniquely correlated well with all three key variables: the fluoroscopic image-based analysis composite score (R2 = 0.40), expert consensus (R2 = 0.76), and performer experience (R2 = 0.41). These results confirm that experts view less efficient technical behavior as indicative of lesser technical proficiency. However, while expert assessments of technical skill were reliable and consistent, neither individual nor consensus expert opinion appears to correlate with performer experience (R2 = 0.11)

AB - Psychomotor skill and decision-making efficiency in surgical wire navigation can be objectively evaluated by analysis of intraoperative fluoroscopic image sequences. Prior work suggests that such image-based behavior analysis of operating room (OR) performance can predict performer experience level (R2 = 0.62) and agree with expert opinion (the current standard) on the quality of a final implant construct (R2 = 0.59). However, it is unclear how objective image-based evaluation compares with expert assessments for entire technical OR performances. This study examines the relationships between three key variables: (1) objective image-based criteria, (2) expert opinions, and (3) performing surgeon experience level. A paired-comparison survey of seven experts done based upon eight OR fluoroscopic wire navigation sequences shows that the experts’ preferences are best explained by objective metrics that reflect psychomotor and decision-making behaviors which are counter-productive to successful implant placement, like image count (R2 = 0.83) and behavior tally (R2 = 0.74). One such behavior, adjustments away from goal, uniquely correlated well with all three key variables: the fluoroscopic image-based analysis composite score (R2 = 0.40), expert consensus (R2 = 0.76), and performer experience (R2 = 0.41). These results confirm that experts view less efficient technical behavior as indicative of lesser technical proficiency. However, while expert assessments of technical skill were reliable and consistent, neither individual nor consensus expert opinion appears to correlate with performer experience (R2 = 0.11)

KW - expert opinion

KW - fluoroscopic image analysis

KW - paired comparison

KW - surgical data science

KW - technical skill assessment

KW - Surgery, Computer-Assisted/methods

KW - Bone Wires

KW - Orthopedic Procedures/methods

UR - http://www.scopus.com/inward/record.url?scp=85172390127&partnerID=8YFLogxK

U2 - 10.1002/jor.25685

DO - 10.1002/jor.25685

M3 - Journal article

C2 - 37652571

SN - 0736-0266

VL - 42

SP - 404

EP - 414

JO - Journal of Orthopaedic Research

JF - Journal of Orthopaedic Research

IS - 2

ER -