PalmGazer: Unimanual Eye-hand Menus in Augmented Reality

TY - GEN

T1 - PalmGazer

T2 - The 2023 ACM Symposium on Spatial User Interaction

AU - Pfeuffer, Ken

AU - Obernolte, Jan

AU - Dietz, Felix

AU - Mäkelä, Ville

AU - Sidenmark, Ludwig

AU - Manakhov, Pavel

AU - Pakanen, Minna

AU - Alt, Florian

PY - 2023/10/13

Y1 - 2023/10/13

N2 - How can we design the user interfaces for augmented reality (AR) so that we can interact as simple, flexible and expressive as we can with smartphones in one hand? To explore this question, we propose PalmGazer as an interaction concept integrating eye-hand interaction to establish a singlehandedly operable menu system. In particular, PalmGazer is designed to support quick and spontaneous digital commands– such as to play a music track, check notifications or browse visual media – through our devised three-way interaction model: hand opening to summon the menu UI, eye-hand input for selection of items, and dragging gesture for navigation. A key aspect is that it remains always-accessible and movable to the user, as the menu supports meaningful hand and head based reference frames. We demonstrate the concept in practice through a prototypical mobile UI with application probes, and describe technique designs specifically-tailored to the application UI. A qualitative evaluation highlights the system’s interaction benefits and drawbacks, e.g., that common 2D scroll and selection tasks are simple to operate, but higher degrees of freedom may be reserved for two hands. Our work contributes interaction techniques and design insights to expand AR’s uni-manual capabilities.

AB - How can we design the user interfaces for augmented reality (AR) so that we can interact as simple, flexible and expressive as we can with smartphones in one hand? To explore this question, we propose PalmGazer as an interaction concept integrating eye-hand interaction to establish a singlehandedly operable menu system. In particular, PalmGazer is designed to support quick and spontaneous digital commands– such as to play a music track, check notifications or browse visual media – through our devised three-way interaction model: hand opening to summon the menu UI, eye-hand input for selection of items, and dragging gesture for navigation. A key aspect is that it remains always-accessible and movable to the user, as the menu supports meaningful hand and head based reference frames. We demonstrate the concept in practice through a prototypical mobile UI with application probes, and describe technique designs specifically-tailored to the application UI. A qualitative evaluation highlights the system’s interaction benefits and drawbacks, e.g., that common 2D scroll and selection tasks are simple to operate, but higher degrees of freedom may be reserved for two hands. Our work contributes interaction techniques and design insights to expand AR’s uni-manual capabilities.

KW - augmented reality

KW - menu

KW - gaze

KW - gestures

KW - eye-hand interaction

UR - http://www.scopus.com/inward/record.url?scp=85176143445&partnerID=8YFLogxK

U2 - 10.1145/3607822.3614523

DO - 10.1145/3607822.3614523

M3 - Article in proceedings

SP - 1

EP - 12

BT - SUI '23

A2 - Huang, Tony

A2 - Sra, Misha

A2 - Argelaguet, Ferran

PB - Association for Computing Machinery

CY - Sydney

Y2 - 13 October 2023 through 15 October 2023

ER -