Rapido: Prototyping Interactive AR Experiences through Programming by Demonstration

Germán Leiva, Jens Emil Grønbæk, Clemens Nylandsted Klokmose, Cuong Nguyen, Rubaiat Habib Kazi, Paul Asente

Research output: Contribution to book/anthology/report/proceedingArticle in proceedingsResearchpeer-review

46 Citations (Scopus)

Abstract

Programming by Demonstration (PbD) is a well-known technique that allows non-programmers to describe interactivity by performing examples of the expected behavior, but it has not been extensively explored for AR. We present Rapido, a novel early-stage prototyping tool to create fully interactive mobile AR prototypes from non-interactive video prototypes using PbD. In Rapido, designers use a mobile AR device to record a video prototype to capture context, sketch assets, and demonstrate interactions. They can demonstrate touch inputs, animation paths, and rules to, e.g., have a sketch follow the focus area of the device or the user's world-space touches. Simultaneously, a live website visualizes an editable overview of all the demonstrated examples and infers a state machine of the user flow. Our key contribution is a method that enables designers to turn a video prototype into an executable state machine through PbD. The designer switches between these representations to interactively refine the final interactive prototype. We illustrate the power of Rapido's approach by prototyping the main interactions of three popular AR mobile applications.

Original languageEnglish
Title of host publicationUIST 2021 - Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology
Number of pages12
PublisherAssociation for Computing Machinery
Publication dateOct 2021
Pages626-637
ISBN (Electronic)9781450386357
DOIs
Publication statusPublished - Oct 2021
Event34th Annual ACM Symposium on User Interface Software and Technology, UIST 2021 - Virtual, Online, United States
Duration: 10 Oct 202114 Oct 2021

Conference

Conference34th Annual ACM Symposium on User Interface Software and Technology, UIST 2021
Country/TerritoryUnited States
CityVirtual, Online
Period10/10/202114/10/2021
SeriesUIST 2021 - Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology

Keywords

  • design by enaction
  • programming with examples
  • rapid prototyping

Fingerprint

Dive into the research topics of 'Rapido: Prototyping Interactive AR Experiences through Programming by Demonstration'. Together they form a unique fingerprint.

Cite this