From inconsistent annotations to ground truth: Aggregation strategies for annotations of proximal carious lesions in dental imagery

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11110%2F25%3A10498394" target="_blank" >RIV/00216208:11110/25:10498394 - isvavai.cz</a>

Výsledek na webu

<a href="https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=uITMAPlqm4" target="_blank" >https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=uITMAPlqm4</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.jdent.2025.105728" target="_blank" >10.1016/j.jdent.2025.105728</a>

Jazyk výsledku

angličtina

Název v původním jazyce

From inconsistent annotations to ground truth: Aggregation strategies for annotations of proximal carious lesions in dental imagery

Popis výsledku v původním jazyce

Objectives: Annotating carious lesions on images is challenging. For artificial intelligence (AI) applications, the aggregation of heterogeneous multi-examiner annotations into one single annotation (e.g. via majority voting, MV) is usually needed. We assessed different aggregation strategies for multi-examiner annotations of primary proximal carious lesions on orthoradial radiographs and Near-Infrared Light Transillumination (NILT) images. Methods: A total of 1007 proximal surfaces from 522 extracted posterior teeth were assessed by five dentists. Histological analysis provided the gold standard. Surfaces were classified as (1) sound, (2) enamel lesion or (3) dentin lesion. Four label aggregation strategies - MV, Weighted Majority Voting (WMV), Dawid-Skene (DS), and multi-annotator competence estimation (MACE) - were applied to unimodal (radiographs, NILT) and multimodal (combined) datasets. The area under the receiver operating characteristic curve (AUROC) was the primary outcome metric. Results: According to the gold standard, 637 (63 %) surfaces were sound, 280 (28 %) showed carious lesions limited to the enamel, and 90 (9 %) showed lesions extending into the dentin. For radiographs, aggregation using MACE outperformed MV, WMV and DS significantly across all lesion depths (p &lt; 0.002). For NILT, MACE significantly outperformed MV across all lesion depths (p &lt; 0.001) and DS for enamel and dentin lesions (p &lt;= 0.002). In the multimodal dataset, DS outperformed the other label aggregation strategies across all lesion depths significantly (p &lt; 0.05). Conclusions: The commonly applied MV may be suboptimal. There is a need for informed application of specific aggregation strategies, depending on the dataset characteristics. Clinical significance: Most AI applications for dental image analysis are trained on a single annotation, usually resulting from aggregated multi-examiner annotations of each image. However, since these annotations are usually aggregated in an in vivo setting where no definitive ground truth is available, the choice of aggregation strategy plays a crucial role.

Název v anglickém jazyce

From inconsistent annotations to ground truth: Aggregation strategies for annotations of proximal carious lesions in dental imagery

Popis výsledku anglicky

Objectives: Annotating carious lesions on images is challenging. For artificial intelligence (AI) applications, the aggregation of heterogeneous multi-examiner annotations into one single annotation (e.g. via majority voting, MV) is usually needed. We assessed different aggregation strategies for multi-examiner annotations of primary proximal carious lesions on orthoradial radiographs and Near-Infrared Light Transillumination (NILT) images. Methods: A total of 1007 proximal surfaces from 522 extracted posterior teeth were assessed by five dentists. Histological analysis provided the gold standard. Surfaces were classified as (1) sound, (2) enamel lesion or (3) dentin lesion. Four label aggregation strategies - MV, Weighted Majority Voting (WMV), Dawid-Skene (DS), and multi-annotator competence estimation (MACE) - were applied to unimodal (radiographs, NILT) and multimodal (combined) datasets. The area under the receiver operating characteristic curve (AUROC) was the primary outcome metric. Results: According to the gold standard, 637 (63 %) surfaces were sound, 280 (28 %) showed carious lesions limited to the enamel, and 90 (9 %) showed lesions extending into the dentin. For radiographs, aggregation using MACE outperformed MV, WMV and DS significantly across all lesion depths (p &lt; 0.002). For NILT, MACE significantly outperformed MV across all lesion depths (p &lt; 0.001) and DS for enamel and dentin lesions (p &lt;= 0.002). In the multimodal dataset, DS outperformed the other label aggregation strategies across all lesion depths significantly (p &lt; 0.05). Conclusions: The commonly applied MV may be suboptimal. There is a need for informed application of specific aggregation strategies, depending on the dataset characteristics. Clinical significance: Most AI applications for dental image analysis are trained on a single annotation, usually resulting from aggregated multi-examiner annotations of each image. However, since these annotations are usually aggregated in an in vivo setting where no definitive ground truth is available, the choice of aggregation strategy plays a crucial role.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

30208 - Dentistry, oral surgery and medicine

Projekt

—

Návaznosti

V - Vyzkumna aktivita podporovana z jinych verejnych zdroju

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Journal of Dentistry

ISSN

0300-5712

e-ISSN

1879-176X

Svazek periodika

157

Číslo periodika v rámci svazku

June

Stát vydavatele periodika

GB - Spojené království Velké Británie a Severního Irska

Počet stran výsledku

8

Strana od-do

105728

Kód UT WoS článku

001474278500001

EID výsledku v databázi Scopus

2-s2.0-105002662277