Let It Flow: Simultaneous Optimization of 3D Flow and Object Clustering

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00376833" target="_blank" >RIV/68407700:21230/25:00376833 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/TIV.2024.3443316" target="_blank" >https://doi.org/10.1109/TIV.2024.3443316</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/TIV.2024.3443316" target="_blank" >10.1109/TIV.2024.3443316</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Let It Flow: Simultaneous Optimization of 3D Flow and Object Clustering

Popis výsledku v původním jazyce

We study the problem of self-supervised 3D scene flow estimation from real large-scale raw point cloud sequences. The problem is crucial to various automotive tasks like trajectory prediction, object detection, and scene reconstruction. In the absence of ground truth scene flow labels, contemporary approaches concentrate on deducing and optimizing flow across sequential pairs of point clouds by incorporating structure-based regularization on flow and object rigidity. The rigid objects are estimated by a variety of 3D spatial clustering methods. While state-of-the-art methods successfully capture overall scene motion using the Neural Prior structure, they encounter challenges in discerning multi-object motions. We identified the structural constraints and the use of large and strict rigid clusters as the main pitfall of the current approaches, and we propose a novel clustering approach that allows for a combination of overlapping soft clusters and non-overlapping rigid clusters. Flow is then jointly estimated with progressively growing non-overlapping rigid clusters together with fixed-size overlapping soft clusters. We evaluate our method on multiple datasets with LiDAR point clouds, demonstrating superior performance over the self-supervised baselines and reaching new state-of-the-art results. Our method excels in resolving flow in complicated dynamic scenes with multiple independently moving objects close to each other, including pedestrians, cyclists, and other vulnerable road users. Our codes are publicly available on https://github.com/ctu-vras/let-it-flow

Název v anglickém jazyce

Let It Flow: Simultaneous Optimization of 3D Flow and Object Clustering

Popis výsledku anglicky

We study the problem of self-supervised 3D scene flow estimation from real large-scale raw point cloud sequences. The problem is crucial to various automotive tasks like trajectory prediction, object detection, and scene reconstruction. In the absence of ground truth scene flow labels, contemporary approaches concentrate on deducing and optimizing flow across sequential pairs of point clouds by incorporating structure-based regularization on flow and object rigidity. The rigid objects are estimated by a variety of 3D spatial clustering methods. While state-of-the-art methods successfully capture overall scene motion using the Neural Prior structure, they encounter challenges in discerning multi-object motions. We identified the structural constraints and the use of large and strict rigid clusters as the main pitfall of the current approaches, and we propose a novel clustering approach that allows for a combination of overlapping soft clusters and non-overlapping rigid clusters. Flow is then jointly estimated with progressively growing non-overlapping rigid clusters together with fixed-size overlapping soft clusters. We evaluate our method on multiple datasets with LiDAR point clouds, demonstrating superior performance over the self-supervised baselines and reaching new state-of-the-art results. Our method excels in resolving flow in complicated dynamic scenes with multiple independently moving objects close to each other, including pedestrians, cyclists, and other vulnerable road users. Our codes are publicly available on https://github.com/ctu-vras/let-it-flow

Druh

J_SC - Článek v periodiku v databázi SCOPUS

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

IEEE Transactions on Intelligent Vehicles

ISSN

2379-8858

e-ISSN

2379-8904

Svazek periodika

10

Číslo periodika v rámci svazku

3

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

9

Strana od-do

2094-2102

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-105013517823