Autonomous aerial cinematography in unstructured environments with learned artistic decision-making

Publikation: Bidrag til tidsskrift/Konferencebidrag i tidsskrift /Bidrag til avisTidsskriftartikelForskningpeer review

DOI

  • Rogerio Bonatti, Carnegie Mellon University
  • ,
  • Wenshan Wang, Carnegie Mellon University
  • ,
  • Cherie Ho, Carnegie Mellon University
  • ,
  • Aayush Ahuja, Carnegie Mellon University
  • ,
  • Mirko Gschwindt, Technical University of Munich
  • ,
  • Efe Camci, Nanyang Technological University
  • ,
  • Erdal Kayacan
  • Sanjiban Choudhury, University of Washington
  • ,
  • Sebastian Scherer, Carnegie Mellon University

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is a demand for an autonomous cinematographer that can reason about both geometry and scene context in real-time. Existing approaches do not address all aspects of this problem; they either require high-precision motion-capture systems or global positioning system tags to localize targets, rely on prior maps of the environment, plan for short time horizons, or only follow fixed artistic guidelines specified before the flight. In this study, we address the problem in its entirety and propose a complete system for real-time aerial cinematography that for the first time combines: (a) vision-based target estimation; (b) 3D signed-distance mapping for occlusion estimation; (c) efficient trajectory optimization for long time-horizon camera motion; and (d) learning-based artistic shot selection. We extensively evaluate our system both in simulation and in field experiments by filming dynamic targets moving through unstructured environments. Our results indicate that our system can operate reliably in the real world without restrictive assumptions. We also provide in-depth analysis and discussions for each module, with the hope that our design tradeoffs can generalize to other related applications. Videos of the complete system can be found at https://youtu.be/ookhHnqmlaU.

OriginalsprogEngelsk
TidsskriftJournal of Field Robotics
Vol/bind37
Nummer4
Sider (fra-til)606-641
Antal sider36
ISSN1556-4959
DOI
StatusUdgivet - jun. 2020

Se relationer på Aarhus Universitet Citationsformater

ID: 189288706