Universal Dependencies according to BERT: both more specific and more general
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F20%3A10424474" target="_blank" >RIV/00216208:11320/20:10424474 - isvavai.cz</a>
Result on the web
<a href="https://www.aclweb.org/anthology/2020.findings-emnlp.245/" target="_blank" >https://www.aclweb.org/anthology/2020.findings-emnlp.245/</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.18653/v1/2020.findings-emnlp.245" target="_blank" >10.18653/v1/2020.findings-emnlp.245</a>
Alternative languages
Result language
angličtina
Original language name
Universal Dependencies according to BERT: both more specific and more general
Original language description
This work focuses on analyzing the form and extent of syntactic abstraction captured by BERT by extracting labeled dependency trees from self-attentions. Previous work showed that individual BERT heads tend to encode particular dependency relation types. We extend these findings by explicitly comparing BERT relations to Universal Dependencies (UD) annotations, showing that they often do not match one-to-one. We suggest a method for relation identification and syntactic tree construction. Our approach produces significantly more consistent dependency trees than previous work, showing that it better explains the syntactic abstractions in BERT. At the same time, it can be successfully applied with only a minimal amount of supervision and generalizes well across languages.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2020
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Findings of the Association for Computational Linguistics: EMNLP 2020
ISBN
978-1-952148-90-3
ISSN
—
e-ISSN
—
Number of pages
13
Pages from-to
2710-2722
Publisher name
Association for Computational Linguistics
Place of publication
Stroudsburg, PA, USA
Event location
Online
Event date
Nov 16, 2020
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—