Multimodal Interaction

TY - CHAP

T1 - Multimodal Interaction

AU - Hoggan, Eve

N1 - Publisher Copyright: © 2025 selection and editorial matter, Constantine Stephanidis and Gavriel Salvendy individual chapters, the contributors.

PY - 2024/1/1

Y1 - 2024/1/1

N2 - Multimodal interfaces enable interaction through multiple sensory channels or modalities, such as visual, auditory, and haptic (touch and gestural). The flexibility of multimodal interfaces allows users to choose an appropriate input or output modality and transition between modalities as necessary to adapt to changing physical contexts and tasks. Multimodal interaction research is divided into two categories: input and output, although truly multimodal interfaces involve both. Multimodal input involves simultaneous, or alternating, use of two or more modalities for user input and processing, offering greater flexibility and expressively powerful options. Multimodal output enables the presentation of information to users through multiple sensory channels or modalities, supporting user accessibility and enabling the use of alternative input and output modalities that can accommodate a wider range of user capabilities. Multimodal interaction research represents a rich and dynamic area of HCI, with numerous potential benefits for users. This chapter delves into the theoretical foundations underlying multimodal interaction research, approaches to designing such interfaces, and examples of the most common modality combinations for input and output. As technological advancements continue to push the boundaries of user interaction, multimodal interaction will become increasingly critical for researchers in HCI.

AB - Multimodal interfaces enable interaction through multiple sensory channels or modalities, such as visual, auditory, and haptic (touch and gestural). The flexibility of multimodal interfaces allows users to choose an appropriate input or output modality and transition between modalities as necessary to adapt to changing physical contexts and tasks. Multimodal interaction research is divided into two categories: input and output, although truly multimodal interfaces involve both. Multimodal input involves simultaneous, or alternating, use of two or more modalities for user input and processing, offering greater flexibility and expressively powerful options. Multimodal output enables the presentation of information to users through multiple sensory channels or modalities, supporting user accessibility and enabling the use of alternative input and output modalities that can accommodate a wider range of user capabilities. Multimodal interaction research represents a rich and dynamic area of HCI, with numerous potential benefits for users. This chapter delves into the theoretical foundations underlying multimodal interaction research, approaches to designing such interfaces, and examples of the most common modality combinations for input and output. As technological advancements continue to push the boundaries of user interaction, multimodal interaction will become increasingly critical for researchers in HCI.

UR - https://www.scopus.com/pages/publications/85203745156

U2 - 10.1201/9781003490678-2

DO - 10.1201/9781003490678-2

M3 - Book chapter

AN - SCOPUS:85203745156

SN - 9781032370033

SP - 45

EP - 63

BT - Interaction Techniques and Technologies in Human-Computer Interaction

PB - CRC Press

CY - Boca Raton

ER -