Developing hand-worn input and haptic support for real-world target finding

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review

  • Yi Ta Hsieh, University of Helsinki
  • ,
  • Antti Jylhä, University of Helsinki
  • ,
  • Valeria Orso, University of Padova
  • ,
  • Salvatore Andolina, Aalto University
  • ,
  • Eve Hoggan
  • Luciano Gamberini, University of Padova
  • ,
  • Giulio Jacucci, Aalto University, University of Helsinki

Locating places in cities is typically facilitated by handheld mobile devices, which draw the visual attention of the user on the screen of the device instead of the surroundings. In this research, we aim at strengthening the connection between people and their surroundings through enabling mid-air gestural interaction with real-world landmarks and delivering information through audio to retain users' visual attention on the scene. Recent research on gesture-based and haptic techniques for such purposes has mainly considered handheld devices that eventually direct users' attention back to the devices. We contribute a hand-worn, mid-air gestural interaction design with directional vibrotactile guidance for finding points of interest (POIs). Through three design iterations, we address aspects of (1) sensing technologies and the placement of actuators considering users' instinctive postures, (2) the feasibility of finding and fetching information regarding landmarks without visual feedback, and (3) the benefits of such interaction in a tourist application. In a final evaluation, participants located POIs and fetched information by pointing and following directional guidance, thus realising a vision in which they found and experienced real-world landmarks while keeping their visual attention on the scene. The results show that the interaction technique has comparable performance to a visual baseline, enables high mobility, and facilitates keeping visual attention on the surroundings.

Original languageEnglish
JournalPersonal and Ubiquitous Computing
Pages (from-to)117-132
Publication statusPublished - Feb 2019

    Research areas

  • Auditory feedback, Augmented reality, Gestural input, Haptic devices, Novel interaction paradigms, Pointing, Ubiquitous and mobile computing design and evaluation

See relations at Aarhus University Citationformats

ID: 135692687