TOPRO: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3ATXSDV93E" target="_blank" >RIV/00216208:11320/25:TXSDV93E - isvavai.cz</a>

Výsledek na webu

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189933264&partnerID=40&md5=6080c6b90adddc9c1614d55e5cc6f5f9" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189933264&partnerID=40&md5=6080c6b90adddc9c1614d55e5cc6f5f9</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

TOPRO: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks

Popis výsledku v původním jazyce

Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily focused on sentence-level classification tasks, and only a few considered token-level labeling tasks such as Named Entity Recognition (NER) and Part-of-Speech (POS) tagging. In this paper, we propose Token-Level Prompt Decomposition (TOPRO), which facilitates the prompt-based method for token-level sequence labeling tasks. The TOPRO method decomposes an input sentence into single tokens and applies one prompt template to each token. Our experiments on multilingual NER and POS tagging datasets demonstrate that TOPRO-based fine-tuning outperforms Vanilla fine-tuning and Prompt-Tuning in zero-shot cross-lingual transfer, especially for languages that are typologically different from the source language English. Our method also attains state-of-the-art performance when employed with the mT5 model. Besides, our exploratory study in multilingual large language models shows that TOPRO performs much better than the current in-context learning method. Overall, the performance improvements show that TOPRO could potentially serve as a novel and simple benchmarking method for sequence labeling tasks. © 2024 Association for Computational Linguistics.

Název v anglickém jazyce

TOPRO: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks

Popis výsledku anglicky

Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily focused on sentence-level classification tasks, and only a few considered token-level labeling tasks such as Named Entity Recognition (NER) and Part-of-Speech (POS) tagging. In this paper, we propose Token-Level Prompt Decomposition (TOPRO), which facilitates the prompt-based method for token-level sequence labeling tasks. The TOPRO method decomposes an input sentence into single tokens and applies one prompt template to each token. Our experiments on multilingual NER and POS tagging datasets demonstrate that TOPRO-based fine-tuning outperforms Vanilla fine-tuning and Prompt-Tuning in zero-shot cross-lingual transfer, especially for languages that are typologically different from the source language English. Our method also attains state-of-the-art performance when employed with the mT5 model. Besides, our exploratory study in multilingual large language models shows that TOPRO performs much better than the current in-context learning method. Overall, the performance improvements show that TOPRO could potentially serve as a novel and simple benchmarking method for sequence labeling tasks. © 2024 Association for Computational Linguistics.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

EACL - Conf. European Chapter Assoc. Comput. Linguist., Proc. Conf.

ISBN

979-889176088-2

ISSN

—

e-ISSN

—

Počet stran výsledku

18

Strana od-do

2685-2702

Název nakladatele

Association for Computational Linguistics (ACL)

Místo vydání

—

Místo konání akce

St. Julian's

Datum konání akce

1. 1. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—