Efficient Use of Large Language Models for Analysis of Text Corpora

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61988987%3A17610%2F24%3AA2502N5Z" target="_blank" >RIV/61988987:17610/24:A2502N5Z - isvavai.cz</a>

Alternative codes found

RIV/68407700:21730/24:00381571

Result on the web

<a href="https://www.scitepress.org/Papers/2024/123498/123498.pdf" target="_blank" >https://www.scitepress.org/Papers/2024/123498/123498.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.5220/0012349800003654" target="_blank" >10.5220/0012349800003654</a>

Result language

angličtina

Original language name

Efficient Use of Large Language Models for Analysis of Text Corpora

Original language description

In this paper, we propose an efficient approach for tracking a given phenomenon in a corpus using natural language processing (NLP) methods. The topic of tracking phenomena in a corpus is important, especially in the fields of sociology, psychology, and economics, which study human behavior in society. Unlike existing approaches that rely on universal large language models (LLMs), which are computationally expensive, we focus on using computationally less expensive methods. These methods allow for high data processing speed while maintaining high accuracy. Our approach is inspired by the cascade approach to optimization, where we first roughly filter out unwanted information and then gradually use more accurate models, which are computationally more expensive. In this way, we are able to process large amounts of data with high accuracy using different models, while also reducing the overall cost of computations. To demonstrate the proposed method, we chose a task that consists of finding the frequency of occurrence of a certain phenomenon in a large text corpus, which is divided into individual months of the year. In practice, this means that we can, for example, use Internet discussions to find out how much people are discussing a particular topic. The entire solution is presented as a pipeline, which consists of individual phases that successively process text data using methods selected to minimize the overall cost of processing all data.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

ICPRAM 2024

ISBN

978-989758684-2

ISSN

2184-4313

e-ISSN

—

Number of pages

11

Pages from-to

695-705

Publisher name

—

Place of publication

Roma, Italy

Event location

Roma, Italy

Event date

Jan 28, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—