Using Game Engine to Generate Synthetic Datasets for Machine Learning

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F20%3A00345199" target="_blank" >RIV/68407700:21230/20:00345199 - isvavai.cz</a>

Výsledek na webu

<a href="https://cescg.org/cescg_submission/using-game-engine-to-generate-synthetic-datasets-for-machine-learning/" target="_blank" >https://cescg.org/cescg_submission/using-game-engine-to-generate-synthetic-datasets-for-machine-learning/</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Using Game Engine to Generate Synthetic Datasets for Machine Learning

Popis výsledku v původním jazyce

Datasets for use in computer vision machine learning areoften challenging to acquire. Often, datasets are createdeither using hand-labeling or via expensive measurements.In this paper, we characterize different augmented imagedata used in computer vision machine learning tasks andpropose a method of generating such data synthetically us-ing a game engine. We implement a Unity plugin for cre-ating such augmented image data outputs, usable in exist-ing Unity projects. The implementation allows for RGBlit output and several ground-truth outputs, such as depthand normal information, object or category segmentation,motion segmentation, forward and backward optical flowand occlusions, 2D and 3D bounding boxes, and cameraparameters. We also explore the possibilities of added re-alism by using an external path-tracing renderer instead ofthe rasterization pipeline, which is currently the standardin most game engines. We demonstrate our tool by cre-ating configurable example scenes, which are specificallydesigned for training machine learning algorithms.

Název v anglickém jazyce

Using Game Engine to Generate Synthetic Datasets for Machine Learning

Popis výsledku anglicky

Datasets for use in computer vision machine learning areoften challenging to acquire. Often, datasets are createdeither using hand-labeling or via expensive measurements.In this paper, we characterize different augmented imagedata used in computer vision machine learning tasks andpropose a method of generating such data synthetically us-ing a game engine. We implement a Unity plugin for cre-ating such augmented image data outputs, usable in exist-ing Unity projects. The implementation allows for RGBlit output and several ground-truth outputs, such as depthand normal information, object or category segmentation,motion segmentation, forward and backward optical flowand occlusions, 2D and 3D bounding boxes, and cameraparameters. We also explore the possibilities of added re-alism by using an external path-tracing renderer instead ofthe rasterization pipeline, which is currently the standardin most game engines. We demonstrate our tool by cre-ating configurable example scenes, which are specificallydesigned for training machine learning algorithms.

Druh

O - Ostatní výsledky

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2020

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů