A Latent-Variable Model for Intrinsic Probing

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A8YZAZEJZ" target="_blank" >RIV/00216208:11320/23:8YZAZEJZ - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85167964610&partnerID=40&md5=412f7f84d56ccb15ddff0b3e37b68d5f" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85167964610&partnerID=40&md5=412f7f84d56ccb15ddff0b3e37b68d5f</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

A Latent-Variable Model for Intrinsic Probing

Original language description

"The success of pre-trained contextualized representations has prompted researchers to analyze them for the presence of linguistic information. Indeed, it is natural to assume that these pre-trained representations do encode some level of linguistic knowledge as they have brought about large empirical improvements on a wide variety of NLP tasks, which suggests they are learning true linguistic generalization. In this work, we focus on intrinsic probing, an analysis technique where the goal is not only to identify whether a representation encodes a linguistic attribute but also to pinpoint where this attribute is encoded. We propose a novel latent-variable formulation for constructing intrinsic probes and derive a tractable variational approximation to the log-likelihood. Our results show that our model is versatile and yields tighter mutual information estimates than two intrinsic probes previously proposed in the literature. Finally, we find empirical evidence that pre-trained representations develop a cross-lingually entangled notion of morphosyntax. Copyright © 2023, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved."

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

"Proc. AAAI Conf. Artif. Intell., AAAI"

ISBN

978-157735880-0

ISSN

—

e-ISSN

—

Number of pages

9

Pages from-to

13591-13599

Publisher name

AAAI Press

Place of publication

—

Event location

Melaka, Malaysia

Event date

Jan 1, 2023

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—