Explainable AI in the military domain

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985955%3A_____%2F24%3A00586451" target="_blank" >RIV/67985955:_____/24:00586451 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1007/s10676-024-09762-w" target="_blank" >https://doi.org/10.1007/s10676-024-09762-w</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s10676-024-09762-w" target="_blank" >10.1007/s10676-024-09762-w</a>

Result language

angličtina

Original language name

Explainable AI in the military domain

Original language description

Artificial intelligence (AI) has become nearly ubiquitous in modern society, from components of mobile applications to medical support systems, and everything in between. In societally impactful systems imbued with AI, there has been increasing concern related to opaque AI, that is, artificial intelligence where it is unclear how or why certain decisions are reached. This has led to a recent boom in research on “explainable AI” (XAI), or approaches to making AI more explainable and understandable to human users. In the military domain, numerous bodies have argued that autonomous and AI-enabled weapon systems ought not incorporate unexplainable AI, with the International Committee of the Red Cross and the United States Department of Defense both explicitly including explainability as a relevant factor in the development and use of such systems. In this article, I present a cautiously critical assessment of this view, arguing that explainability will be irrelevant for many current and near-future autonomous systems in the military (which do not incorporate any AI), that it will be trivially incorporated into most military systems which do possess AI (as these generally possess simpler AI systems), and that for those systems with genuinely opaque AI, explainability will prove to be of more limited value than one might imagine. In particular, I argue that explainability, while indeed a virtue in design, is a virtue aimed primarily at designers and troubleshooters of AI-enabled systems, but is far less relevant for users and handlers actually deploying these systems. I further argue that human-machine teaming is a far more important element of responsibly using AI for military purposes, adding that explainability may undermine efforts to improve human-machine teamings by creating a prima facie sense that the AI, due to its explainability, may be utilized with little (or less) potential for mistakes. I conclude by clarifying that the arguments are not against XAI in the military, but are instead intended as a caution against over-inflating the value of XAI in this domain, or ignoring the limitations and potential pitfalls of this approach.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

60302 - Ethics (except ethics related to specific subfields)

Project

<a href="/en/project/GN24-12638I" target="_blank" >GN24-12638I: Ethical and Legal Aspects of Autonomous Weapon Systems</a><br>

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Ethics and Information Technology

ISSN

1388-1957

e-ISSN

1572-8439

Volume of the periodical

26

Issue of the periodical within the volume

2

Country of publishing house

NL - THE KINGDOM OF THE NETHERLANDS

Number of pages

13

Pages from-to

29

UT code for WoS article

001203946100001

EID of the result in the Scopus database

2-s2.0-85190401777