Prompt. Exploit. Repeat: Automating Network Security Testing with LLMs

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00383332" target="_blank" >RIV/68407700:21230/25:00383332 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1007/978-3-031-87330-0_2" target="_blank" >https://doi.org/10.1007/978-3-031-87330-0_2</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-031-87330-0_2" target="_blank" >10.1007/978-3-031-87330-0_2</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Prompt. Exploit. Repeat: Automating Network Security Testing with LLMs

Popis výsledku v původním jazyce

Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, but their potential in cybersecurity remains largely unexplored. Despite their inherent limitations, LLM-based designs have shown promising ability in planning and navigating open-world scenarios. This paper investigates the application of pre-trained LLMs as agents in network security environments, a domain traditionally dominated by reinforcement learning (RL) approaches. We introduce a novel method that leverages LLMs for sequential decision-making in cybersecurity scenarios, that is flexible and does not need re-training to adapt to new scenarios. Our study employs two distinct environments: Microsoft’s CyberBattleSim and our newly developed NetSecGame, which incorporates more realistic network conditions and a defender component. We compare the performance of LLM agents, against traditional reinforcement learning agents across various scenarios. Results show that the best LLM agents achieve success rates of 100% in undefended scenarios and up to 53.3% in the most challenging defended scenarios, outperforming conventional RL agents without requiring additional training. Furthermore, we present NetSecGame, a modular and scalable network security environment that addresses the limitations of existing platforms by providing more realistic conditions for testing attacking and defending agents. This research demonstrates the potential of LLMs in cybersecurity applications, offering a flexible and efficient alternative to traditional RL approaches in network security testing.

Název v anglickém jazyce

Prompt. Exploit. Repeat: Automating Network Security Testing with LLMs

Popis výsledku anglicky

Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, but their potential in cybersecurity remains largely unexplored. Despite their inherent limitations, LLM-based designs have shown promising ability in planning and navigating open-world scenarios. This paper investigates the application of pre-trained LLMs as agents in network security environments, a domain traditionally dominated by reinforcement learning (RL) approaches. We introduce a novel method that leverages LLMs for sequential decision-making in cybersecurity scenarios, that is flexible and does not need re-training to adapt to new scenarios. Our study employs two distinct environments: Microsoft’s CyberBattleSim and our newly developed NetSecGame, which incorporates more realistic network conditions and a defender component. We compare the performance of LLM agents, against traditional reinforcement learning agents across various scenarios. Results show that the best LLM agents achieve success rates of 100% in undefended scenarios and up to 53.3% in the most challenging defended scenarios, outperforming conventional RL agents without requiring additional training. Furthermore, we present NetSecGame, a modular and scalable network security environment that addresses the limitations of existing platforms by providing more realistic conditions for testing attacking and defending agents. This research demonstrates the potential of LLMs in cybersecurity applications, offering a flexible and efficient alternative to traditional RL approaches in network security testing.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/VJ02010020" target="_blank" >VJ02010020: AI-Dojo: Multiagentní testbed pro výzkum a testování umělé inteligence v kyberbezpečnosti</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Agents and Artificial Intelligence

ISBN

978-3-031-87329-4

ISSN

0302-9743

e-ISSN

—

Počet stran výsledku

22

Strana od-do

15-36

Název nakladatele

Springer, Cham

Místo vydání

—

Místo konání akce

Rome

Datum konání akce

24. 2. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—