ESPnet-SpeechLM: An Open Speech Language Model Toolkit

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0201388" target="_blank" >RIV/00216305:26230/26:0201388 - isvavai.cz</a>

Result on the web

<a href="https://aclanthology.org/2025.naacl-demo.12.pdf" target="_blank" >https://aclanthology.org/2025.naacl-demo.12.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.18653/v1/2025.naacl-demo.12" target="_blank" >10.18653/v1/2025.naacl-demo.12</a>

Result language

angličtina

Original language name

ESPnet-SpeechLM: An Open Speech Language Model Toolkit

Original language description

We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs) and voice-driven agentic applications. The toolkit standardizes speech processing tasks by framing them as universal sequential modeling problems, encompassing a cohesive workflow of data preprocessing, pre-training, inference, and task evaluation. With ESPnet-SpeechLM, users can easily define task templates and configure key settings, enabling seamless and streamlined SpeechLM development. The toolkit ensures flexibility, efficiency, and scalability by offering highly configurable modules for every stage of the workflow. To illustrate its capabilities, we provide multiple use cases demonstrating how competitive SpeechLMs can be constructed with ESPnet-SpeechLM, including a 1.7B-parameter model pre-trained on both text and speech tasks, across diverse benchmarks.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

S - Specificky vyzkum na vysokych skolach

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies: Long Papers, NAACL-HLT 2025

ISBN

9798891761919

ISSN

—

e-ISSN

—

Number of pages

9

Pages from-to

116-124

Publisher name

Association for Computational Linguistics (ACL)

Place of publication

Hybrid, Albuquerque, New Mexico, USA

Event location

Hybrid, Albuquerque, NM, USA

Event date

Apr 29, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

001598875800012