R-BPE: Improving BPE-Tokenizers with Token Reuse
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3A54936S35" target="_blank" >RIV/00216208:11320/26:54936S35 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2025.emnlp-main.1169/" target="_blank" >https://aclanthology.org/2025.emnlp-main.1169/</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
R-BPE: Improving BPE-Tokenizers with Token Reuse
Original language description
This paper presents R-BPE, a lightweight framework for adapting existing Byte-Pair Encoding (BPE) tokenizers to better support a specified target language. It reuses tokens from user-excluded languages and creates ID-based maps to resolve the new tokens of the chosen language. We evaluate R-BPE on Arabic as a target language. R-BPE reduced subword fertility by an average of 24.4% across the LLaMA 3.1 8B, Command R 35B, and Qwen 3 8B models. Applied to LLaMA 3.1 8B in continued pretraining mode, R-BPE yields a 7.33% reduction in training time. On the ArabicMMLU benchmark, the resulting model improved by 5.09 points on five in-domain topics and matched the original model’s overall performance. It also preserved performance on EnglishMMLU.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
ISBN
979-8-89176-332-6
ISSN
—
e-ISSN
—
Number of pages
9
Pages from-to
22951-22959
Publisher name
—
Place of publication
—
Event location
Suzhou, China
Event date
Jan 1, 2026
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—