ResidualViT for Efficient Temporally Dense Video Encoding

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F25%3A00388532" target="_blank" >RIV/68407700:21730/25:00388532 - isvavai.cz</a>

Result on the web

<a href="https://openaccess.thecvf.com/content/ICCV2025/papers/Soldan_ResidualViT_for_Efficient_Temporally_Dense_Video_Encoding_ICCV_2025_paper.pdf" target="_blank" >https://openaccess.thecvf.com/content/ICCV2025/papers/Soldan_ResidualViT_for_Efficient_Temporally_Dense_Video_Encoding_ICCV_2025_paper.pdf</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

ResidualViT for Efficient Temporally Dense Video Encoding

Original language description

Several video understanding tasks, such as natural language temporal video grounding, temporal activity localization, and audio description generation, require "temporally dense" reasoning over frames sampled at high temporal resolution. However, computing frame-level features for these tasks is computationally expensive given the temporal resolution requirements. In this paper, we make three contributions to reduce the cost of computing features for temporally dense tasks. First, we introduce a vision transformer (ViT) architecture, dubbed ResidualViT, that leverages the large temporal redundancy in videos to efficiently compute temporally dense frame-level features. Our architecture incorporates (i) learnable residual connections that ensure temporal consistency across consecutive frames and (ii) a token reduction module that enhances processing speed by selectively discarding temporally redundant information while reusing weights of a pretrained foundation model. Second, we propose a lightweight distillation strategy to approximate the frame-level features of the original foundation model. Finally, we evaluate our approach across four tasks and five datasets, in both zero-shot and fully supervised settings, demonstrating significant reductions in computational cost (up to ) and improvements in inference speed (up to faster), all while closely approximating the accuracy of the original foundation model.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

N - Vyzkumna aktivita podporovana z neverejnych zdroju

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

ICCV2025: Proceedings of the International Conference on Computer Vision

ISBN

—

ISSN

1550-5499

e-ISSN

—

Number of pages

11

Pages from-to

22305-22315

Publisher name

IEEE Communications Society

Place of publication

Anchorage

Event location

Honolulu

Event date

Oct 19, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—