Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AC57QSC4S" target="_blank" >RIV/00216208:11320/25:C57QSC4S - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85195990752&partnerID=40&md5=adfb8f1d72229b5dc324450eb21bff07" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85195990752&partnerID=40&md5=adfb8f1d72229b5dc324450eb21bff07</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer

Original language description

Unsupervised cross-lingual transfer involves transferring knowledge between languages without explicit supervision. Although numerous studies have been conducted to improve performance in such tasks by focusing on cross-lingual knowledge, particularly lexical and syntactic knowledge, current approaches are limited as they only incorporate syntactic or lexical information. Since each type of information offers unique advantages and no previous attempts have combined both, we attempt to explore the potential of this approach. In this paper, we present a novel framework called”Lexicon-Syntax Enhanced Multilingual BERT” that combines both lexical and syntactic knowledge. Specifically, we use Multilingual BERT (mBERT) as the base model and employ two techniques to enhance its learning capabilities. The code-switching technique is used to implicitly teach the model lexical alignment information, while a syntactic-based graph attention network is designed to help the model encode syntactic structure. To integrate both types of knowledge, we input code-switched sequences into both the syntactic module and the mBERT base model simultaneously. Our extensive experimental results demonstrate this framework can consistently outperform all baselines of zero-shot cross-lingual transfer, with the gains of 1.0∼3.7 points on text classification, named entity recognition (ner), and semantic parsing tasks. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Jt. Int. Conf. Comput. Linguist., Lang. Resour. Eval., LREC-COLING - Main Conf. Proc.

ISBN

978-249381410-4

ISSN

—

e-ISSN

—

Number of pages

12

Pages from-to

8986-8997

Publisher name

European Language Resources Association (ELRA)

Place of publication

—

Event location

Torino, Italia

Event date

Jan 1, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—