“You’ll be a nurse, my son!” Automatically assessing gender biases in autoregressive language models in French and Italian

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3AZH9U529Q" target="_blank" >RIV/00216208:11320/26:ZH9U529Q - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1007/s10579-024-09780-6" target="_blank" >http://dx.doi.org/10.1007/s10579-024-09780-6</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s10579-024-09780-6" target="_blank" >10.1007/s10579-024-09780-6</a>

Jazyk výsledku

angličtina

Název v původním jazyce

“You’ll be a nurse, my son!” Automatically assessing gender biases in autoregressive language models in French and Italian

Popis výsledku v původním jazyce

Language models are now massively used for a variety of tasks, including open-ended generation and writing assistance. However, generated texts can encapsulate biases and harm users. A variety of articles aim at detecting, measuring and mitigating stereotypical biases, but focus mainly on English and on pre-training tasks. Thus, we propose a framework to automatically measure gender biases generated by language models in inflected languages, in a practical setting. Herein, we report experiments using this framework on seven autoregressive language models used to generate more than 52,000 cover letters in French, addressing 203 industry and sectors, and over 4100 cover letters in Italian, on 55 sectors. Associations between occupation and gender are studied using a system that we introduce to automatically identify morpho-syntactic gender markers in text. Results suggest that all models are strongly biased towards the generation of texts containing masculine gender markers. Overall, generated texts contain twice as many masculine (vs. feminine) markers in French, and eight times as many in Italian. Models also exacerbate gender stereotypes that are evidenced in social science studies and associate feminine inflections with occupations related to care, children and physical appearance, whereas occupations that require physical, technical and manual skills are strongly associated with masculine markers. © The Author(s), under exclusive licence to Springer Nature B.V. 2024.

Název v anglickém jazyce

“You’ll be a nurse, my son!” Automatically assessing gender biases in autoregressive language models in French and Italian

Popis výsledku anglicky

Language models are now massively used for a variety of tasks, including open-ended generation and writing assistance. However, generated texts can encapsulate biases and harm users. A variety of articles aim at detecting, measuring and mitigating stereotypical biases, but focus mainly on English and on pre-training tasks. Thus, we propose a framework to automatically measure gender biases generated by language models in inflected languages, in a practical setting. Herein, we report experiments using this framework on seven autoregressive language models used to generate more than 52,000 cover letters in French, addressing 203 industry and sectors, and over 4100 cover letters in Italian, on 55 sectors. Associations between occupation and gender are studied using a system that we introduce to automatically identify morpho-syntactic gender markers in text. Results suggest that all models are strongly biased towards the generation of texts containing masculine gender markers. Overall, generated texts contain twice as many masculine (vs. feminine) markers in French, and eight times as many in Italian. Models also exacerbate gender stereotypes that are evidenced in social science studies and associate feminine inflections with occupations related to care, children and physical appearance, whereas occupations that require physical, technical and manual skills are strongly associated with masculine markers. © The Author(s), under exclusive licence to Springer Nature B.V. 2024.

Druh

J_SC - Článek v periodiku v databázi SCOPUS

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Language Resources and Evaluation

ISSN

1574-020X

e-ISSN

—

Svazek periodika

59

Číslo periodika v rámci svazku

2

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

29

Strana od-do

1495-1523

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-85207686245