HPLT's Second Data Release
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10511569" target="_blank" >RIV/00216208:11320/25:10511569 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2025.mtsummit-2.21/" target="_blank" >https://aclanthology.org/2025.mtsummit-2.21/</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
HPLT's Second Data Release
Original language description
We describe the progress of the High Performance Language Technologies (HPLT) project, a 3-year EU-funded project that started in September 2022. We focus on the up-to-date results on the release of free text datasets derived from web crawls, one of the central objectives of the project. The second release used a revised processing pipeline, and an enlarged set of input crawls. From 4.5 petabytes of web crawls we extracted 7.6T tokens of monolingual text in 193 languages, plus 380 million parallel sentences in 51 language pairs. We also release MultiHPLT, a cross-combination of the parallel data, which produces 1,275 pairs, as well as releasing the containing documents for all parallel sentences in order to enable research in document-level MT. We report changes in the pipeline, analysis and evaluation results for the second parallel data release based on machine translation systems. All datasets are released under a permissive CC0 licence.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of Machine Translation Summit XX: Volume 2
ISBN
978-2-9701897-1-8
ISSN
—
e-ISSN
—
Number of pages
2
Pages from-to
101-102
Publisher name
European Association for Machine Translation (EAMT)
Place of publication
Genève, Switzerland
Event location
Genève, Switzerland
Event date
Jun 23, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—