PreCNet: Next-Frame Video Prediction Based on Predictive Coding

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F24%3A00365341" target="_blank" >RIV/68407700:21230/24:00365341 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/TNNLS.2023.3240857" target="_blank" >https://doi.org/10.1109/TNNLS.2023.3240857</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/TNNLS.2023.3240857" target="_blank" >10.1109/TNNLS.2023.3240857</a>

Jazyk výsledku

angličtina

Název v původním jazyce

PreCNet: Next-Frame Video Prediction Based on Predictive Coding

Popis výsledku v původním jazyce

Predictive coding, currently a highly influential theory in neuroscience, has not been widely adopted in machine learning yet. In this work, we transform the seminal model of Rao and Ballard (1999) into a modern deep learning framework while remaining maximally faithful to the original schema. The resulting network we propose (PreCNet) is tested on a widely used next frame video prediction benchmark, which consists of images from an urban environment recorded from a car-mounted camera, and achieves state-of-the-art performance. Performance on all measures (MSE, PSNR, SSIM) was further improved when a larger training set (2M images from BDD100k), pointing to the limitations of the KITTI training set. This work demonstrates that an architecture carefully based in a neuroscience model, without being explicitly tailored to the task at hand, can exhibit exceptional performance.

Název v anglickém jazyce

PreCNet: Next-Frame Video Prediction Based on Predictive Coding

Popis výsledku anglicky

Predictive coding, currently a highly influential theory in neuroscience, has not been widely adopted in machine learning yet. In this work, we transform the seminal model of Rao and Ballard (1999) into a modern deep learning framework while remaining maximally faithful to the original schema. The resulting network we propose (PreCNet) is tested on a widely used next frame video prediction benchmark, which consists of images from an urban environment recorded from a car-mounted camera, and achieves state-of-the-art performance. Performance on all measures (MSE, PSNR, SSIM) was further improved when a larger training set (2M images from BDD100k), pointing to the limitations of the KITTI training set. This work demonstrates that an architecture carefully based in a neuroscience model, without being explicitly tailored to the task at hand, can exhibit exceptional performance.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

IEEE Transactions on Neural Networks and Learning Systems

ISSN

2162-237X

e-ISSN

2162-2388

Svazek periodika

35

Číslo periodika v rámci svazku

8

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

15

Strana od-do

10353-10367

Kód UT WoS článku

000932859200001

EID výsledku v databázi Scopus

2-s2.0-85148422306