Ask the experts: sourcing a high-quality nutrition counseling dataset through Human-AI collaboration

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F24%3A10492847" target="_blank" >RIV/00216208:11320/24:10492847 - isvavai.cz</a>

Result on the web

<a href="https://aclanthology.org/2024.findings-emnlp.674/" target="_blank" >https://aclanthology.org/2024.findings-emnlp.674/</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.18653/v1/2024.findings-emnlp.674" target="_blank" >10.18653/v1/2024.findings-emnlp.674</a>

Result language

angličtina

Original language name

Ask the experts: sourcing a high-quality nutrition counseling dataset through Human-AI collaboration

Original language description

Large Language Models (LLMs) are being employed by end-users for various tasks, including sensitive ones such as health counseling, disregarding potential safety concerns. It is thus necessary to understand how adequately LLMs perform in such domains. We conduct a case study on ChatGPT in nutrition counseling, a popular use-case where the model supports a user with their dietary struggles. We crowdsource real-world diet-related struggles, then work with nutrition experts to generate supportive text using ChatGPT. Finally, experts evaluate the safety and text quality of ChatGPT&apos;s output. The result is the HAI-Coaching dataset, containing ~2.4K crowdsourced dietary struggles and ~97K corresponding ChatGPT-generated and expert-annotated supportive texts. We analyse ChatGPT&apos;s performance, discovering potentially harmful behaviours, especially for sensitive topics like mental health. Finally, we use HAI-Coaching to test open LLMs on various downstream tasks, showing that even the latest models struggle to

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

R - Projekt Ramcoveho programu EK

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Findings of the Association for Computational Linguistics: EMNLP 2024

ISBN

979-8-89176-168-1

ISSN

—

e-ISSN

—

Number of pages

27

Pages from-to

11519-11545

Publisher name

Association for Computational Linguistics

Place of publication

Kerrville, TX, USA

Event location

Miami, FL, USA

Event date

Nov 12, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—