From inconsistent annotations to ground truth: Aggregation strategies for annotations of proximal carious lesions in dental imagery

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11110%2F25%3A10498394" target="_blank" >RIV/00216208:11110/25:10498394 - isvavai.cz</a>

Result on the web

<a href="https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=uITMAPlqm4" target="_blank" >https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=uITMAPlqm4</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.jdent.2025.105728" target="_blank" >10.1016/j.jdent.2025.105728</a>

Result language

angličtina

Original language name

From inconsistent annotations to ground truth: Aggregation strategies for annotations of proximal carious lesions in dental imagery

Original language description

Objectives: Annotating carious lesions on images is challenging. For artificial intelligence (AI) applications, the aggregation of heterogeneous multi-examiner annotations into one single annotation (e.g. via majority voting, MV) is usually needed. We assessed different aggregation strategies for multi-examiner annotations of primary proximal carious lesions on orthoradial radiographs and Near-Infrared Light Transillumination (NILT) images. Methods: A total of 1007 proximal surfaces from 522 extracted posterior teeth were assessed by five dentists. Histological analysis provided the gold standard. Surfaces were classified as (1) sound, (2) enamel lesion or (3) dentin lesion. Four label aggregation strategies - MV, Weighted Majority Voting (WMV), Dawid-Skene (DS), and multi-annotator competence estimation (MACE) - were applied to unimodal (radiographs, NILT) and multimodal (combined) datasets. The area under the receiver operating characteristic curve (AUROC) was the primary outcome metric. Results: According to the gold standard, 637 (63 %) surfaces were sound, 280 (28 %) showed carious lesions limited to the enamel, and 90 (9 %) showed lesions extending into the dentin. For radiographs, aggregation using MACE outperformed MV, WMV and DS significantly across all lesion depths (p &lt; 0.002). For NILT, MACE significantly outperformed MV across all lesion depths (p &lt; 0.001) and DS for enamel and dentin lesions (p &lt;= 0.002). In the multimodal dataset, DS outperformed the other label aggregation strategies across all lesion depths significantly (p &lt; 0.05). Conclusions: The commonly applied MV may be suboptimal. There is a need for informed application of specific aggregation strategies, depending on the dataset characteristics. Clinical significance: Most AI applications for dental image analysis are trained on a single annotation, usually resulting from aggregated multi-examiner annotations of each image. However, since these annotations are usually aggregated in an in vivo setting where no definitive ground truth is available, the choice of aggregation strategy plays a crucial role.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

30208 - Dentistry, oral surgery and medicine

Project

—

Continuities

V - Vyzkumna aktivita podporovana z jinych verejnych zdroju

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Journal of Dentistry

ISSN

0300-5712

e-ISSN

1879-176X

Volume of the periodical

157

Issue of the periodical within the volume

June

Country of publishing house

GB - UNITED KINGDOM

Number of pages

8

Pages from-to

105728

UT code for WoS article

001474278500001

EID of the result in the Scopus database

2-s2.0-105002662277