Video Search with CLIP and Interactive Text Query Reformulation

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A10468887" target="_blank" >RIV/00216208:11320/23:10468887 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1007/978-3-031-27077-2_50" target="_blank" >https://doi.org/10.1007/978-3-031-27077-2_50</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-031-27077-2_50" target="_blank" >10.1007/978-3-031-27077-2_50</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Video Search with CLIP and Interactive Text Query Reformulation

Popis výsledku v původním jazyce

Nowadays, deep learning based models like CLIP allow simple design of cross-modal video search systems that are able to solve many tasks considered as highly challenging several years ago. In this paper, we analyze a CLIP based search approach that focuses on situations, where users cannot find proper text queries to describe searched video segments. The approach relies on suggestions of classes for displayed intermediate result sets and thus allows users to realize missing words and ideas to describe video frames. This approach is supported with a preliminary study showing potential of the method. Based on the results, we extend a respected known-item search system for the Video Browser Showdown, where more challenging visual known-item search tasks are planned.

Název v anglickém jazyce

Video Search with CLIP and Interactive Text Query Reformulation

Popis výsledku anglicky

Nowadays, deep learning based models like CLIP allow simple design of cross-modal video search systems that are able to solve many tasks considered as highly challenging several years ago. In this paper, we analyze a CLIP based search approach that focuses on situations, where users cannot find proper text queries to describe searched video segments. The approach relies on suggestions of classes for displayed intermediate result sets and thus allows users to realize missing words and ideas to describe video frames. This approach is supported with a preliminary study showing potential of the method. Based on the results, we extend a respected known-item search system for the Video Browser Showdown, where more challenging visual known-item search tasks are planned.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

MULTIMEDIA MODELING, MMM 2023, PT I

ISBN

978-3-031-27076-5

ISSN

0302-9743

e-ISSN

1611-3349

Počet stran výsledku

6

Strana od-do

628-633

Název nakladatele

SPRINGER INTERNATIONAL PUBLISHING AG

Místo vydání

CHAM

Místo konání akce

Bergen

Datum konání akce

9. 1. 2023

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000996563000050