Linguistic Rule Induction Improves Adversarial and OOD Robustness in Large Language Models

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3ATDED4A2K" target="_blank" >RIV/00216208:11320/25:TDED4A2K - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85195915375&partnerID=40&md5=16d68f89b42f03ed02b1e1775dbf6a57" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85195915375&partnerID=40&md5=16d68f89b42f03ed02b1e1775dbf6a57</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Linguistic Rule Induction Improves Adversarial and OOD Robustness in Large Language Models

Original language description

Ensuring robustness is especially important when AI is deployed in responsible or safety-critical environments. ChatGPT can perform brilliantly in both adversarial and out-of-distribution (OOD) robustness. Still, other popular large language models (LLMs), like LLaMA-2, ERNIE, and ChatGLM, do not perform satisfactorily in this regard. Therefore, it is valuable to study what efforts play essential roles in ChatGPT, and how to transfer these efforts to other LLMs. This paper experimentally finds that linguistic rule induction is the foundation for identifying the cause-effect relationships in LLMs. Accurately processing the cause-effect relationships in LLMs can improve their adversarial and OOD robustness. Furthermore, we explore a low-cost way of aligning LLMs with linguistic rules. Specifically, we constructed a linguistic rule instruction dataset to fine-tune LLMs. To further energize LLMs for reasoning step-by-step with the linguistic rules, we propose the task-relevant LingR-based chain-of-thoughts. Experiments showed that LingR-induced LLaMA-13B achieves comparable or better results with GPT-3.5 and GPT-4 on various adversarial and OOD robustness evaluations. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Jt. Int. Conf. Comput. Linguist., Lang. Resour. Eval., LREC-COLING - Main Conf. Proc.

ISBN

978-249381410-4

ISSN

—

e-ISSN

—

Number of pages

13

Pages from-to

10565-10577

Publisher name

European Language Resources Association (ELRA)

Place of publication

—

Event location

Torino, Italia

Event date

Jan 1, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—