HPLT’s First Release of Data and Models
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F24%3A10492846" target="_blank" >RIV/00216208:11320/24:10492846 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2024.eamt-2.27/" target="_blank" >https://aclanthology.org/2024.eamt-2.27/</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
HPLT’s First Release of Data and Models
Original language description
The High Performance Language Technologies (HPLT) project is a 3-year EU-funded project that started in September 2022. It aims to deliver free, sustainable, and reusable datasets, models, and workflows at scale using high-performance computing. We describe the first results of the project. The data release includes monolingual data in 75 languages at 5.6T tokens and parallel data in 18 language pairs at 96M pairs, derived from 1.8 petabytes of web crawls. Building upon automated and transparent pipelines, the first machine translation (MT) models as well as large language models (LLMs) have been trained and released. Multiple data processing tools and pipelines have also been made public.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 2)
ISBN
978-1-06-869071-6
ISSN
—
e-ISSN
—
Number of pages
2
Pages from-to
53-54
Publisher name
European Association for Machine Translation (EAMT)
Place of publication
Sheffield, UK
Event location
Sheffield, UK
Event date
Jun 24, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—