Creating 3D Diorama from Single Image with Deep Learning
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A10486946" target="_blank" >RIV/00216208:11320/23:10486946 - isvavai.cz</a>
Result on the web
<a href="https://ceur-ws.org/Vol-3498/paper3.pdf" target="_blank" >https://ceur-ws.org/Vol-3498/paper3.pdf</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Creating 3D Diorama from Single Image with Deep Learning
Original language description
Creating 3D scenes is a time-consuming task that requires experience with modeling software. This paper presents a novel approach that combines neural models for panoptic segmentation and monocular depth estimation to construct dioramas. While previous research has explored generating dioramas from single images, to the best of our knowledge, there is no research utilizing deep learning techniques for the task. This paper provides an analysis of existing approaches to diorama generation. We then describe the construction of the diorama, where objects identified by segmentation are separated intodistinct images with transparent backgrounds. These images are then placed in a 3D scene, arranged to reflect the estimated depth of each object. We also address several challenges that had to be overcome. Specifically, we employed fine-tuning to address the limitations of the available depth model when applied to outdoor scenes. Our method has been implemented as an add-on for the open-source 3D software Blender, utilizing neural models in the ONNX format for depth and segmentation inferences.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2023
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
CEUR Workshop Proceedings
ISBN
—
ISSN
1613-0073
e-ISSN
1613-0073
Number of pages
11
Pages from-to
25-35
Publisher name
CEUR-WS
Place of publication
Tatranske Matliare, Slovakia
Event location
Tatranské Matliare, Slovakia
Event date
Sep 22, 2023
Type of event by nationality
EUR - Evropská akce
UT code for WoS article
—