3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing

Haoran Li, Long Ma, Haolin Shi, Yanbin Hao, Yong Liao*, Lechao Cheng, Peng Yuan Zhou*

*Corresponding author af dette arbejde

Publikation: Bidrag til bog/antologi/rapport/proceedingKonferencebidrag i proceedingsForskningpeer review

Abstract

The current GAN inversion methods typically can only edit the appearance and shape of a single object and background while overlooking spatial information. In this work, we propose a 3D editing framework, 3D-GOI  to enable multifaceted editing of affine information (scale, translation, and rotation) on multiple objects. 3D-GOI realizes the complex editing function by inverting the abundance of attribute codes (object shape/ appearance/ scale/ rotation/ translation, background shape/ appearance, and camera pose) controlled by GIRAFFE, a renowned 3D GAN. Accurately inverting all the codes is challenging, 3D-GOI solves this challenge following three main steps. First, we segment the objects and the background in a multi-object image. Second, we use a custom Neural Inversion Encoder to obtain coarse codes of each object. Finally, we use a round-robin optimization algorithm to get precise codes to reconstruct the image. To the best of our knowledge, 3D-GOI is the first framework to enable multifaceted editing on multiple objects. Both qualitative and quantitative experiments demonstrate that 3D-GOI holds immense potential for flexible, multifaceted editing in complex multi-object scenes. Our project and code are released at https://3d-goi.github.io.

OriginalsprogEngelsk
TitelComputer Vision – ECCV 2024 - 18th European Conference, Proceedings
RedaktørerAleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, Gül Varol
Antal sider17
ForlagSpringer Science and Business Media Deutschland GmbH
Publikationsdato2025
Sider390-406
ISBN (Trykt)9783031730320
DOI
StatusUdgivet - 2025
Begivenhed18th European Conference on Computer Vision, ECCV 2024 - Milan, Italien
Varighed: 29 sep. 20244 okt. 2024

Konference

Konference18th European Conference on Computer Vision, ECCV 2024
Land/OmrådeItalien
ByMilan
Periode29/09/202404/10/2024
NavnLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Vol/bind15120 LNCS
ISSN0302-9743

Fingeraftryk

Dyk ned i forskningsemnerne om '3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing'. Sammen danner de et unikt fingeraftryk.

Citationsformater