Hands-Free VR
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61988987%3A17310%2F25%3AA2603BLZ" target="_blank" >RIV/61988987:17310/25:A2603BLZ - isvavai.cz</a>
Result on the web
<a href="https://www.scitepress.org/Link.aspx?doi=10.5220/0013115100003912" target="_blank" >https://www.scitepress.org/Link.aspx?doi=10.5220/0013115100003912</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.5220/0013115100003912" target="_blank" >10.5220/0013115100003912</a>
Alternative languages
Result language
angličtina
Original language name
Hands-Free VR
Original language description
The paper introduces Hands-Free VR, a voice-based natural-language interface for VR. The user gives a command using their voice, the speech audio data is converted to text using a speech-to-text deep learning model that is fine-tuned for robustness to word phonetic similarity and to spoken English accents, and the text is mapped to an executable VR command using a large language model that is robust to natural language diversity. Hands-Free VR was evaluated in a controlled within-subjects study (N = 22) that asked participants to find specific objects and to place them in various configurations. In the control condition participants used a conventional VR user interface to grab, carry, and position the objects using the handheld controllers. In the experimental condition participants used Hands-Free VR. The results confirm that: (1) Hands-Free VR is robust to spoken English accents, as for 20 of our participants English was not their first language, and to word phonetic similarity, correctly transcribing the voice command 96.71% of the time; (2) Hands-Free VR is robust to natural language diversity, correctly mapping the transcribed command to an executable command in 97.83% of the time; (3) Hands-Free VR had a significant efficiency advantage over the conventional VR interface in terms of task completion time, total viewpoint translation, total view direction rotation, and total left and right hand translations; (4) Hands-Free VR received high user preference ratings in terms of ease of use, intuitiveness, ergonomics, reliability, and desirability.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10200 - Computer and information sciences
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - HUCAPP
ISBN
978-989-758-728-3
ISSN
2184-4321
e-ISSN
—
Number of pages
10
Pages from-to
533-542
Publisher name
Science and Technology Publications, Lda
Place of publication
Porto
Event location
Porto
Event date
Feb 26, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—