Through the compression glass: Language complexity and the linguistic structure of compressed strings

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AF47LZDBR" target="_blank" >RIV/00216208:11320/25:F47LZDBR - isvavai.cz</a>

Výsledek na webu

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189963136&doi=10.1515%2flingvan-2022-0140&partnerID=40&md5=8003d7dfa187e357a2eefae9b41e2ae1" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189963136&doi=10.1515%2flingvan-2022-0140&partnerID=40&md5=8003d7dfa187e357a2eefae9b41e2ae1</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1515/lingvan-2022-0140" target="_blank" >10.1515/lingvan-2022-0140</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Through the compression glass: Language complexity and the linguistic structure of compressed strings

Popis výsledku v původním jazyce

Against the backdrop of the sociolinguistic-typological complexity debate which is all about measuring, comparing and explaining language complexity, this article investigates how Kolmogorov-based information theoretic complexity relates to linguistic structures. Specifically, the linguistic structure of text which has been compressed with the text compression algorithm gzip will be analysed. One implementation of Kolmogorov-based language complexity is the compression technique (Ehret, Katharina. 2021. An information-theoretic view on language complexity and register variation: Compressing naturalistic corpus data. Corpus Linguistics and Linguistic Theory (2). 383-410) which employs gzip to measure language complexity in naturalistic text samples. In order to determine what type of structures compression algorithms like gzip capture, and how these compressed strings relate to linguistically meaningful structures, gzip's lexicon output is retrieved and subjected to an in-depth analysis. As a case study, the compression technique is applied to the English version of Lewis Carroll's Alice's Adventures in Wonderland and its lexicon output is extracted. The results show that gzip-like algorithms sometimes capture linguistically meaningful structures which coincide, for instance, with lexical words or suffixes. However, many compressed sequences are linguistically unintelligible or simply do not coincide with any linguistically meaningful structures. Compression algorithms like gzip thus crucially capture purely formal structural regularities. As a consequence, information theoretic complexity, in this context, is a linguistically agnostic, purely structural measure of regularity and redundancy in texts. © 2024 the author(s), published by De Gruyter, Berlin/Boston 2024.

Název v anglickém jazyce

Through the compression glass: Language complexity and the linguistic structure of compressed strings

Popis výsledku anglicky

Against the backdrop of the sociolinguistic-typological complexity debate which is all about measuring, comparing and explaining language complexity, this article investigates how Kolmogorov-based information theoretic complexity relates to linguistic structures. Specifically, the linguistic structure of text which has been compressed with the text compression algorithm gzip will be analysed. One implementation of Kolmogorov-based language complexity is the compression technique (Ehret, Katharina. 2021. An information-theoretic view on language complexity and register variation: Compressing naturalistic corpus data. Corpus Linguistics and Linguistic Theory (2). 383-410) which employs gzip to measure language complexity in naturalistic text samples. In order to determine what type of structures compression algorithms like gzip capture, and how these compressed strings relate to linguistically meaningful structures, gzip's lexicon output is retrieved and subjected to an in-depth analysis. As a case study, the compression technique is applied to the English version of Lewis Carroll's Alice's Adventures in Wonderland and its lexicon output is extracted. The results show that gzip-like algorithms sometimes capture linguistically meaningful structures which coincide, for instance, with lexical words or suffixes. However, many compressed sequences are linguistically unintelligible or simply do not coincide with any linguistically meaningful structures. Compression algorithms like gzip thus crucially capture purely formal structural regularities. As a consequence, information theoretic complexity, in this context, is a linguistically agnostic, purely structural measure of regularity and redundancy in texts. © 2024 the author(s), published by De Gruyter, Berlin/Boston 2024.

Druh

J_SC - Článek v periodiku v databázi SCOPUS

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Linguistics Vanguard

ISSN

2199-174X

e-ISSN

—

Svazek periodika

2024

Číslo periodika v rámci svazku

2024

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

11

Strana od-do

1-11

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-85189963136