Effectiveness of Combinatorial Interaction Testing in Test Automation - An Industrial Case Study

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00388886" target="_blank" >RIV/68407700:21230/25:00388886 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1145/3756681.3757016" target="_blank" >https://doi.org/10.1145/3756681.3757016</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3756681.3757016" target="_blank" >10.1145/3756681.3757016</a>

Result language

angličtina

Original language name

Effectiveness of Combinatorial Interaction Testing in Test Automation - An Industrial Case Study

Original language description

Combinatorial Interaction Testing (CIT) is an established software testing method having a wide application area. While many studies have been conducted to investigate the effectiveness of CIT through various simulations and mutation testing, there needs to be more evidence regarding its effectiveness in real industrial testing processes. In response to this certain gap, this work investigates the practical implications of CIT application in a real software project with real historical defects as well as artificial defects created to resemble historical defects. The evidence presented encompasses two distinct studies: firstly, the optimization of test data input into user interface forms, and secondly, the management of system configurations for testing purposes. These studies employ the Tricentis Test Automation for ServiceNow (TTA-SNOW) outcome as the system under evaluation. We cautiously evaluated the impact of CIT tool on various metrics, such as time spent by test design, test automation and test execution, test suite size, and defect detection compared to an intuitive approach taken by test engineers without proper application of CIT. Our findings demonstrate that CIT has the potential to reduce test suite size and execution time while maintaining or improving defect detection rates. Specifically, in Study 1, CIT reduced average defect detection time from 5.11 hours to 3.39 hours. In Study 2, focusing on system configurations, this time was reduced from 4.38 hours to 2.28 hours. These findings provide valuable insights into CIT's practical benefits in industrial contexts.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

EASE '25: Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering

ISBN

979-8-4007-1385-9

ISSN

—

e-ISSN

—

Number of pages

7

Pages from-to

659-665

Publisher name

ACM Press

Place of publication

New York

Event location

Istanbul

Event date

Jun 17, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

001668832700062