Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F22%3A00365086" target="_blank" >RIV/68407700:21230/22:00365086 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/CVPR52688.2022.01552" target="_blank" >https://doi.org/10.1109/CVPR52688.2022.01552</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/CVPR52688.2022.01552" target="_blank" >10.1109/CVPR52688.2022.01552</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos

Popis výsledku v původním jazyce

We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. To this end, we model the blurred appearance of a fast moving object in a generative fashion by parametrizing its 3D position, rotation, velocity, acceleration, bounces, shape, and texture over the duration of a predefined time window spanning multiple frames. Using differentiable rendering, we are able to estimate all parameters by minimizing the pixel-wise reprojection error to the input video via backpropagating through a rendering pipeline that accounts for motion blur by averaging the graphics output over short time intervals. For that purpose, we also estimate the camera exposure gap time within the same optimization. To account for abrupt motion changes like bounces, we model the motion trajectory as a piece-wise polynomial, and we are able to estimate the specific time of the bounce at sub-frame accuracy. Experiments on established benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.

Název v anglickém jazyce

Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos

Popis výsledku anglicky

We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. To this end, we model the blurred appearance of a fast moving object in a generative fashion by parametrizing its 3D position, rotation, velocity, acceleration, bounces, shape, and texture over the duration of a predefined time window spanning multiple frames. Using differentiable rendering, we are able to estimate all parameters by minimizing the pixel-wise reprojection error to the input video via backpropagating through a rendering pipeline that accounts for motion blur by averaging the graphics output over short time intervals. For that purpose, we also estimate the camera exposure gap time within the same optimization. To account for abrupt motion changes like bounces, we model the motion trajectory as a piece-wise polynomial, and we are able to estimate the specific time of the bounce at sub-frame accuracy. Experiments on established benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EF16_019%2F0000765" target="_blank" >EF16_019/0000765: Výzkumné centrum informatiky</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceeding 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

ISBN

978-1-6654-6946-3

ISSN

1063-6919

e-ISSN

2575-7075

Počet stran výsledku

10

Strana od-do

15969-15978

Název nakladatele

IEEE

Místo vydání

Piscataway

Místo konání akce

New Orleans, Louisiana

Datum konání akce

19. 6. 2022

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000870783001075