3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing

Haoran Li; Long Ma; Haolin Shi; Yanbin Hao; Yong Liao; Lechao Cheng; Peng Yuan Zhou

doi:10.1007/978-3-031-73033-7_22

3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing

Haoran Li, Long Ma, Haolin Shi, Yanbin Hao, Yong Liao^*, Lechao Cheng, Peng Yuan Zhou^*

^*Corresponding author for this work

Department of Electrical and Computer Engineering - Communication, Control and Automation

Research output: Contribution to book/anthology/report/proceeding › Article in proceedings › Research › peer-review

Abstract

The current GAN inversion methods typically can only edit the appearance and shape of a single object and background while overlooking spatial information. In this work, we propose a 3D editing framework, 3D-GOI to enable multifaceted editing of affine information (scale, translation, and rotation) on multiple objects. 3D-GOI realizes the complex editing function by inverting the abundance of attribute codes (object shape/ appearance/ scale/ rotation/ translation, background shape/ appearance, and camera pose) controlled by GIRAFFE, a renowned 3D GAN. Accurately inverting all the codes is challenging, 3D-GOI solves this challenge following three main steps. First, we segment the objects and the background in a multi-object image. Second, we use a custom Neural Inversion Encoder to obtain coarse codes of each object. Finally, we use a round-robin optimization algorithm to get precise codes to reconstruct the image. To the best of our knowledge, 3D-GOI is the first framework to enable multifaceted editing on multiple objects. Both qualitative and quantitative experiments demonstrate that 3D-GOI holds immense potential for flexible, multifaceted editing in complex multi-object scenes. Our project and code are released at https://3d-goi.github.io.

Original language	English
Title of host publication	Computer Vision – ECCV 2024 - 18th European Conference, Proceedings
Editors	Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, Gül Varol
Number of pages	17
Publisher	Springer Science and Business Media Deutschland GmbH
Publication date	2025
Pages	390-406
ISBN (Print)	9783031730320
DOIs	https://doi.org/10.1007/978-3-031-73033-7_22
Publication status	Published - 2025
Event	18th European Conference on Computer Vision, ECCV 2024 - Milan, Italy Duration: 29 Sept 2024 → 4 Oct 2024

Conference

Conference	18th European Conference on Computer Vision, ECCV 2024
Country/Territory	Italy
City	Milan
Period	29/09/2024 → 04/10/2024

Series	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	15120 LNCS
ISSN	0302-9743

Access to Document

10.1007/978-3-031-73033-7_22

Cite this

Li, H., Ma, L., Shi, H., Hao, Y., Liao, Y., Cheng, L., & Zhou, P. Y. (2025). 3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing. In A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, & G. Varol (Eds.), Computer Vision – ECCV 2024 - 18th European Conference, Proceedings (pp. 390-406). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-73033-7_22

Li, Haoran ; Ma, Long ; Shi, Haolin et al. / 3D-GOI : 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing. Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. editor / Aleš Leonardis ; Elisa Ricci ; Stefan Roth ; Olga Russakovsky ; Torsten Sattler ; Gül Varol. Springer Science and Business Media Deutschland GmbH, 2025. pp. 390-406 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 15120 LNCS).

@inproceedings{8527285063534ce79cfae5dd0e6f801c,

title = "3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing",

abstract = "The current GAN inversion methods typically can only edit the appearance and shape of a single object and background while overlooking spatial information. In this work, we propose a 3D editing framework, 3D-GOI to enable multifaceted editing of affine information (scale, translation, and rotation) on multiple objects. 3D-GOI realizes the complex editing function by inverting the abundance of attribute codes (object shape/ appearance/ scale/ rotation/ translation, background shape/ appearance, and camera pose) controlled by GIRAFFE, a renowned 3D GAN. Accurately inverting all the codes is challenging, 3D-GOI solves this challenge following three main steps. First, we segment the objects and the background in a multi-object image. Second, we use a custom Neural Inversion Encoder to obtain coarse codes of each object. Finally, we use a round-robin optimization algorithm to get precise codes to reconstruct the image. To the best of our knowledge, 3D-GOI is the first framework to enable multifaceted editing on multiple objects. Both qualitative and quantitative experiments demonstrate that 3D-GOI holds immense potential for flexible, multifaceted editing in complex multi-object scenes. Our project and code are released at https://3d-goi.github.io.",

author = "Haoran Li and Long Ma and Haolin Shi and Yanbin Hao and Yong Liao and Lechao Cheng and Zhou, {Peng Yuan}",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.; 18th European Conference on Computer Vision, ECCV 2024 ; Conference date: 29-09-2024 Through 04-10-2024",

year = "2025",

doi = "10.1007/978-3-031-73033-7_22",

language = "English",

isbn = "9783031730320",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "390--406",

editor = "Ale{\v s} Leonardis and Elisa Ricci and Stefan Roth and Olga Russakovsky and Torsten Sattler and G{\"u}l Varol",

booktitle = "Computer Vision – ECCV 2024 - 18th European Conference, Proceedings",

address = "Germany",

}

Li, H, Ma, L, Shi, H, Hao, Y, Liao, Y, Cheng, L & Zhou, PY 2025, 3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing. in A Leonardis, E Ricci, S Roth, O Russakovsky, T Sattler & G Varol (eds), Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. Springer Science and Business Media Deutschland GmbH, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 15120 LNCS, pp. 390-406, 18th European Conference on Computer Vision, ECCV 2024, Milan, Italy, 29/09/2024. https://doi.org/10.1007/978-3-031-73033-7_22

3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing. / Li, Haoran; Ma, Long; Shi, Haolin et al.
Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. ed. / Aleš Leonardis; Elisa Ricci; Stefan Roth; Olga Russakovsky; Torsten Sattler; Gül Varol. Springer Science and Business Media Deutschland GmbH, 2025. p. 390-406 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 15120 LNCS).

Research output: Contribution to book/anthology/report/proceeding › Article in proceedings › Research › peer-review

TY - GEN

T1 - 3D-GOI

T2 - 18th European Conference on Computer Vision, ECCV 2024

AU - Li, Haoran

AU - Ma, Long

AU - Shi, Haolin

AU - Hao, Yanbin

AU - Liao, Yong

AU - Cheng, Lechao

AU - Zhou, Peng Yuan

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

PY - 2025

Y1 - 2025

N2 - The current GAN inversion methods typically can only edit the appearance and shape of a single object and background while overlooking spatial information. In this work, we propose a 3D editing framework, 3D-GOI to enable multifaceted editing of affine information (scale, translation, and rotation) on multiple objects. 3D-GOI realizes the complex editing function by inverting the abundance of attribute codes (object shape/ appearance/ scale/ rotation/ translation, background shape/ appearance, and camera pose) controlled by GIRAFFE, a renowned 3D GAN. Accurately inverting all the codes is challenging, 3D-GOI solves this challenge following three main steps. First, we segment the objects and the background in a multi-object image. Second, we use a custom Neural Inversion Encoder to obtain coarse codes of each object. Finally, we use a round-robin optimization algorithm to get precise codes to reconstruct the image. To the best of our knowledge, 3D-GOI is the first framework to enable multifaceted editing on multiple objects. Both qualitative and quantitative experiments demonstrate that 3D-GOI holds immense potential for flexible, multifaceted editing in complex multi-object scenes. Our project and code are released at https://3d-goi.github.io.

AB - The current GAN inversion methods typically can only edit the appearance and shape of a single object and background while overlooking spatial information. In this work, we propose a 3D editing framework, 3D-GOI to enable multifaceted editing of affine information (scale, translation, and rotation) on multiple objects. 3D-GOI realizes the complex editing function by inverting the abundance of attribute codes (object shape/ appearance/ scale/ rotation/ translation, background shape/ appearance, and camera pose) controlled by GIRAFFE, a renowned 3D GAN. Accurately inverting all the codes is challenging, 3D-GOI solves this challenge following three main steps. First, we segment the objects and the background in a multi-object image. Second, we use a custom Neural Inversion Encoder to obtain coarse codes of each object. Finally, we use a round-robin optimization algorithm to get precise codes to reconstruct the image. To the best of our knowledge, 3D-GOI is the first framework to enable multifaceted editing on multiple objects. Both qualitative and quantitative experiments demonstrate that 3D-GOI holds immense potential for flexible, multifaceted editing in complex multi-object scenes. Our project and code are released at https://3d-goi.github.io.

UR - http://www.scopus.com/inward/record.url?scp=85208575820&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-73033-7_22

DO - 10.1007/978-3-031-73033-7_22

M3 - Article in proceedings

AN - SCOPUS:85208575820

SN - 9783031730320

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 390

EP - 406

BT - Computer Vision – ECCV 2024 - 18th European Conference, Proceedings

A2 - Leonardis, Aleš

A2 - Ricci, Elisa

A2 - Roth, Stefan

A2 - Russakovsky, Olga

A2 - Sattler, Torsten

A2 - Varol, Gül

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 29 September 2024 through 4 October 2024

ER -

Li H, Ma L, Shi H, Hao Y, Liao Y, Cheng L et al. 3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing. In Leonardis A, Ricci E, Roth S, Russakovsky O, Sattler T, Varol G, editors, Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2025. p. 390-406. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 15120 LNCS). doi: 10.1007/978-3-031-73033-7_22

3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this