Eye-Hand Movement of Objects in Near Space Extended Reality

TY - GEN

T1 - Eye-Hand Movement of Objects in Near Space Extended Reality

AU - Wagner, Uta

AU - Jacobsen, Andreas Asferg

AU - Feuchtner, Tiare

AU - Gellersen, Hans

AU - Pfeuffer, Ken

N1 - Publisher Copyright: © 2024 Owner/Author.

PY - 2024/10

Y1 - 2024/10

N2 - Hand-tracking in Extended Reality (XR) enables moving objects in near space with direct hand gestures, to pick, drag and drop objects in 3D. In this work, we investigate the use of eye-tracking to reduce the effort involved in this interaction. As the eyes naturally look ahead to the target for a drag operation, the principal idea is to map the translation of the object in the image plane to gaze, such that the hand only needs to control the depth component of the operation. We have implemented four techniques that explore two factors: the use of gaze only to move objects in X-Y vs. extra refinement by hand, and the use of hand input in the Z axis to directly move objects vs. indirectly via a transfer function. We compared all four techniques in a user study (N=24) against baselines of direct and indirect hand input. We detail user performance, effort and experience trade-offs and show that all eye-hand techniques significantly reduce physical effort over direct gestures, pointing toward effortless drag-and-drop for XR environments.

AB - Hand-tracking in Extended Reality (XR) enables moving objects in near space with direct hand gestures, to pick, drag and drop objects in 3D. In this work, we investigate the use of eye-tracking to reduce the effort involved in this interaction. As the eyes naturally look ahead to the target for a drag operation, the principal idea is to map the translation of the object in the image plane to gaze, such that the hand only needs to control the depth component of the operation. We have implemented four techniques that explore two factors: the use of gaze only to move objects in X-Y vs. extra refinement by hand, and the use of hand input in the Z axis to directly move objects vs. indirectly via a transfer function. We compared all four techniques in a user study (N=24) against baselines of direct and indirect hand input. We detail user performance, effort and experience trade-offs and show that all eye-hand techniques significantly reduce physical effort over direct gestures, pointing toward effortless drag-and-drop for XR environments.

KW - draganddrop

KW - eye-tracking

KW - gaze interaction

KW - near space

KW - object manipulation

UR - https://www.scopus.com/pages/publications/85215068220

U2 - 10.1145/3654777.3676446

DO - 10.1145/3654777.3676446

M3 - Article in proceedings

AN - SCOPUS:85215068220

BT - UIST 2024 - Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

PB - Association for Computing Machinery

T2 - 37th Annual ACM Symposium on User Interface Software and Technology, UIST 2024

Y2 - 13 October 2024 through 16 October 2024

ER -