Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68378122%3A_____%2F23%3A00579735" target="_blank" >RIV/68378122:_____/23:00579735 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.3233/FAIA230965" target="_blank" >http://dx.doi.org/10.3233/FAIA230965</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.3233/FAIA230965" target="_blank" >10.3233/FAIA230965</a>
Alternative languages
Result language
angličtina
Original language name
Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies
Original language description
Thematic analysis and other variants of inductive coding are widely used qualitative analytic methods within empirical legal studies (ELS). We propose a novel framework facilitating effective collaboration of a legal expert with a large language model (LLM) for generating initial codes (phase 2 of thematic analysis), searching for themes (phase 3), and classifying the data in terms of the themes (to kick-start phase 4). We employed the framework for an analysis of a dataset (n = 785) of facts descriptions from criminal court opinions regarding thefts. The goal of the analysis was to discover classes of typical thefts. Our results show that the LLM, namely OpenAI’s GPT-4, generated reasonable initial codes, and it was capable of improving the quality of the codes based on expert feedback. They also suggest that the model performed well in zero-shot classification of facts descriptions in terms of the themes. Finally, the themes autonomously discovered by the LLM appear to map fairly well to the themes arrived at by legal experts. These findings can be leveraged by legal researchers to guide their decisions in integrating LLMs into their thematic analyses, as well as other inductive coding projects.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
50501 - Law
Result continuities
Project
<a href="/en/project/GA19-15077S" target="_blank" >GA19-15077S: Sentencing disparities in the post-communist continental legal systems</a><br>
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2023
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Legal Knowledge and Information Systems
ISBN
978-1-64368-364-5
ISSN
—
e-ISSN
—
Number of pages
10
Pages from-to
197-206
Publisher name
IOS Press
Place of publication
Amsterdam
Event location
Maastricht
Event date
Dec 18, 2023
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—