Bilinear Image Translation for Temporal Analysis of Photo Collections

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F21%3A00340392" target="_blank" >RIV/68407700:21730/21:00340392 - isvavai.cz</a>
Result on the web
<a href="https://doi.org/10.1109/TPAMI.2019.2950317" target="_blank" >https://doi.org/10.1109/TPAMI.2019.2950317</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1109/TPAMI.2019.2950317" target="_blank" >10.1109/TPAMI.2019.2950317</a>

Result language
angličtina
Original language name
Bilinear Image Translation for Temporal Analysis of Photo Collections
Original language description
We propose an approach for analyzing unpaired visual data annotated with timestamps by generating how images would have looked like if they were from different times. To isolate and transfer time-dependent appearance variations, we introduce a new trainable bilinear factor separation module. We analyze its relation to classical factored representations and concatenation-based auto-encoders. We demonstrate this new module has clear advantages compared to standard concatenation when used in a bottleneck encoder-decoder convolutional neural network architecture. We also show that it can be inserted in a recent adversarial image translation architecture, enabling transfer to multiple different target time periods using a single network. We apply our model to a challenging collection of more than 13,000 cars manufactured between 1920 and 2000 and a dataset of high school yearbook portraits from 1930 to 2009. This allows us, for a given new input image, to generate a "history-lapse video" revealing changes over time by simply varying the latent variable corresponding to time. We show that by analyzing the generated history-lapse videos we can identify object deformations across time, extracting interesting changes in visual style over decades.
Czech name
—
Czech description
—

Type
J<sub>imp</sub> - Article in a specialist periodical, which is included in the Web of Science database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project
<a href="/en/project/EF15_003%2F0000468" target="_blank" >EF15_003/0000468: Intelligent Machine Perception</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2021
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical
IEEE Transactions on Pattern Analysis and Machine Intelligence
ISSN
0162-8828
e-ISSN
1939-3539
Volume of the periodical
43
Issue of the periodical within the volume
4
Country of publishing house
US - UNITED STATES
Number of pages
16
Pages from-to
1197-1212
UT code for WoS article
000626525300007
EID of the result in the Scopus database
2-s2.0-85102238237

Similar results(10)