Aarhus University Seal

huSync - A model and system for the measure of synchronization in small groups: A case study on musical joint action

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review

  • Sanket Rajeev Sabharwal, University of Genoa
  • ,
  • Manuel Varlet, Western Sydney University
  • ,
  • Matthew Breaden, Western Sydney University
  • ,
  • Gualtiero Volpe, University of Genoa
  • ,
  • Antonio Camurri, University of Genoa
  • ,
  • Peter E. Keller

Human communication entails subtle non-verbal modes of expression, which can be analyzed quantitatively using computational approaches and thus support human sciences. In this paper we present huSync, a computational framework and system that utilizes trajectory information extracted using pose estimation algorithms from video sequences to quantify synchronization between individuals in small groups. The system is exploited to study interpersonal coordination in musical ensembles. Musicians communicate with each other through sounds and gestures, providing nonverbal cues that regulate interpersonal coordination. huSync was applied to recordings of concert performances by a professional instrumental ensemble playing two musical pieces. We examined effects of different aspects of musical structure (texture and phrase position) on interpersonal synchronization, which was quantified by computing phase locking values of head motion for all possible within-group pairs. Results indicate that interpersonal coupling was stronger for polyphonic textures (ambiguous leadership) than homophonic textures (clear melodic leader), and this difference was greater in early portions of phrases than endings (where coordination demands are highest). Results were cross-validated against an analysis of audio features, showing links between phase locking values and event density. This research produced a system, huSync, that can quantify synchronization in small groups and is sensitive to dynamic modulations of interpersonal coupling related to ambiguity in leadership and coordination demands, in standard video recordings of naturalistic human group interaction. huSync enabled a better understanding of the relationship between interpersonal coupling and musical structure, thus enhancing collaborations between human and computer scientists.

Original languageEnglish
JournalIEEE Access
Volume10
Pages (from-to)92357-92372
Number of pages16
ISSN2169-3536
DOIs
Publication statusPublished - 2022

    Research areas

  • Behavioral sciences, Couplings, Entrainment, Interpersonal synchronization, Joint actions, Leadership, Music, Musical ensemble performance, Nonverbal communication, Pose estimation, Social factors, Social interaction, Social signal processing, Synchronization, Video recording

See relations at Aarhus University Citationformats

ID: 282389872