Bilinear Image Translation for Temporal Analysis of Photo Collections

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F21%3A00340392" target="_blank" >RIV/68407700:21730/21:00340392 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/TPAMI.2019.2950317" target="_blank" >https://doi.org/10.1109/TPAMI.2019.2950317</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/TPAMI.2019.2950317" target="_blank" >10.1109/TPAMI.2019.2950317</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Bilinear Image Translation for Temporal Analysis of Photo Collections

Popis výsledku v původním jazyce

We propose an approach for analyzing unpaired visual data annotated with timestamps by generating how images would have looked like if they were from different times. To isolate and transfer time-dependent appearance variations, we introduce a new trainable bilinear factor separation module. We analyze its relation to classical factored representations and concatenation-based auto-encoders. We demonstrate this new module has clear advantages compared to standard concatenation when used in a bottleneck encoder-decoder convolutional neural network architecture. We also show that it can be inserted in a recent adversarial image translation architecture, enabling transfer to multiple different target time periods using a single network. We apply our model to a challenging collection of more than 13,000 cars manufactured between 1920 and 2000 and a dataset of high school yearbook portraits from 1930 to 2009. This allows us, for a given new input image, to generate a "history-lapse video" revealing changes over time by simply varying the latent variable corresponding to time. We show that by analyzing the generated history-lapse videos we can identify object deformations across time, extracting interesting changes in visual style over decades.

Název v anglickém jazyce

Bilinear Image Translation for Temporal Analysis of Photo Collections

Popis výsledku anglicky

We propose an approach for analyzing unpaired visual data annotated with timestamps by generating how images would have looked like if they were from different times. To isolate and transfer time-dependent appearance variations, we introduce a new trainable bilinear factor separation module. We analyze its relation to classical factored representations and concatenation-based auto-encoders. We demonstrate this new module has clear advantages compared to standard concatenation when used in a bottleneck encoder-decoder convolutional neural network architecture. We also show that it can be inserted in a recent adversarial image translation architecture, enabling transfer to multiple different target time periods using a single network. We apply our model to a challenging collection of more than 13,000 cars manufactured between 1920 and 2000 and a dataset of high school yearbook portraits from 1930 to 2009. This allows us, for a given new input image, to generate a "history-lapse video" revealing changes over time by simply varying the latent variable corresponding to time. We show that by analyzing the generated history-lapse videos we can identify object deformations across time, extracting interesting changes in visual style over decades.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EF15_003%2F0000468" target="_blank" >EF15_003/0000468: Inteligentní strojové vnímání</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2021

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

IEEE Transactions on Pattern Analysis and Machine Intelligence

ISSN

0162-8828

e-ISSN

1939-3539

Svazek periodika

43

Číslo periodika v rámci svazku

4

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

16

Strana od-do

1197-1212

Kód UT WoS článku

000626525300007

EID výsledku v databázi Scopus

2-s2.0-85102238237