Analyzing BERT's Knowledge of Hypernymy via Prompting

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F21%3A10440538" target="_blank" >RIV/00216208:11320/21:10440538 - isvavai.cz</a>

Result on the web

<a href="https://aclanthology.org/2021.blackboxnlp-1.20.pdf" target="_blank" >https://aclanthology.org/2021.blackboxnlp-1.20.pdf</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Analyzing BERT's Knowledge of Hypernymy via Prompting

Original language description

The high performance of large pretrained language models (LLMs) such as BERT (Devlin et al., 2019) on NLP tasks has prompted questions about BERT&apos;s linguistic capabilities, and how they differ from humans&apos;. In this paper, we approach this question by examining BERT&apos;s knowledge of lexical semantic relations. We focus on hypernymy, the &quot;is-a&quot; relation that relates a word to a superordinate category. We use a prompting methodology to simply ask BERT what the hypernym of a given word is. We find that, in a setting where all hypernyms are guessable via prompting, BERT knows hypernyms with up to 57% accuracy. Moreover, BERT with prompting outperforms other unsupervised models for hypernym discovery even in an unconstrained scenario. However, BERT&apos;s predictions and performance on a dataset containing uncommon hyponyms and hypernyms indicate that its knowledge of hypernymy is still limited.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GX19-26934X" target="_blank" >GX19-26934X: Neural Representations in Multi-modal and Multi-lingual Modeling</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2021

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 4th Workshop on Analyzing and Interpreting Neural Networks for NLP

ISBN

978-1-955917-06-3

ISSN

—

e-ISSN

—

Number of pages

8

Pages from-to

275-282

Publisher name

Association for Computational Linguistics

Place of publication

Stroudsburg, PA, USA

Event location

Punta Cana, Dominican Republic

Event date

Nov 11, 2021

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—