ACCURACY EVALUATION AND ERROR ANALYSIS OF DEPENDENCY PARSING FOR TEXTS IN UKRAINIAN

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3A5VZ9MNLD" target="_blank" >RIV/00216208:11320/26:5VZ9MNLD - isvavai.cz</a>

Result on the web

<a href="http://dx.doi.org/10.30837/2522-9818.2025.2.102" target="_blank" >http://dx.doi.org/10.30837/2522-9818.2025.2.102</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.30837/2522-9818.2025.2.102" target="_blank" >10.30837/2522-9818.2025.2.102</a>

Result language

angličtina

Original language name

ACCURACY EVALUATION AND ERROR ANALYSIS OF DEPENDENCY PARSING FOR TEXTS IN UKRAINIAN

Original language description

The subject of our research is the dependency parsing of sentences in the Ukrainian language using the Universal Dependencies framework. The goal of the work is to evaluate the accuracy of existing transition-based and graph-based parsing architectures with and without deep word embeddings on the Ukrainian dataset, and to analyze the error profiles of such parsers. The article addresses two tasks. One is to evaluate the accuracy of several modern dependency parsing approaches applied to a hand-annotated gold standard dataset, using labeled and unlabeled attachment scores as the metric to evaluate the parsing accuracy. The other task is to analyze and categorize the errors made by standard parsers. Resolving these errors could potentially allow us to build a more accurate parser in the future. Error rate for different categories is compared to the ba seline error rate, and statistical significance of such comparison is validated using the chi-square method. The key results are as follows. For the Ukrainian language, parsing accuracy is greatly increased with the use of deep word embeddings. Transition-based parser with deep word embeddings provides the highest labeled attachment score of 84.66% for the test dataset. For the same parser, higher error rates are associated with non-projectivity of dependencies, higher sentence length and higher distance to head. Also, for pronouns and numerals the error rate for labeled attachment is significantly higher than the baseline, while the unlabeled error rate is at the baseline. Conclusions: parsing accuracy for the Ukrainian dataset is sub-par in comparison with other languages, but the overall trend of accuracy improvement with the use of deep word embeddings is consistent with existing research. To improve overall parsing accuracy, we must focus on such problem areas as non-projective dependencies, longer sentences, and greater distance between the head and the dependent. In future work we intend to explore ways to improve parsing accuracy by supplementing neural parsing with other approaches, like formal rules or pre-and post-processing. © K. Syrotkin, 2025.

Czech name

—

Czech description

—

Type

J_SC - Article in a specialist periodical, which is included in the SCOPUS database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Innovative Technologies and Scientific Solutions for Industries

ISSN

2522-9818

e-ISSN

—

Volume of the periodical

2025

Issue of the periodical within the volume

2

Country of publishing house

US - UNITED STATES

Number of pages

9

Pages from-to

102-110

UT code for WoS article

—

EID of the result in the Scopus database

2-s2.0-105019070600