AI generates covertly racist decisions about people based on their dialect

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AE9XNTWPC" target="_blank" >RIV/00216208:11320/25:E9XNTWPC - isvavai.cz</a>

Výsledek na webu

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85202506726&doi=10.1038%2fs41586-024-07856-5&partnerID=40&md5=82cfa7237f4d8030765c93eaf3050d1b" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85202506726&doi=10.1038%2fs41586-024-07856-5&partnerID=40&md5=82cfa7237f4d8030765c93eaf3050d1b</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1038/s41586-024-07856-5" target="_blank" >10.1038/s41586-024-07856-5</a>

Jazyk výsledku

angličtina

Název v původním jazyce

AI generates covertly racist decisions about people based on their dialect

Popis výsledku v původním jazyce

Hundreds of millions of people now interact with language models, with uses ranging from help with writing1,2 to informing hiring decisions3. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans4–7. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement8,9. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology. © The Author(s) 2024.

Název v anglickém jazyce

AI generates covertly racist decisions about people based on their dialect

Popis výsledku anglicky

Hundreds of millions of people now interact with language models, with uses ranging from help with writing1,2 to informing hiring decisions3. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans4–7. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement8,9. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology. © The Author(s) 2024.

Druh

J_SC - Článek v periodiku v databázi SCOPUS

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Nature

ISSN

0028-0836

e-ISSN

—

Svazek periodika

633

Číslo periodika v rámci svazku

8028

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

8

Strana od-do

147-154

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-85202506726