Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0197861" target="_blank" >RIV/00216305:26230/26:0197861 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1016/j.patrec.2025.04.012" target="_blank" >https://doi.org/10.1016/j.patrec.2025.04.012</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.patrec.2025.04.012" target="_blank" >10.1016/j.patrec.2025.04.012</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks

Popis výsledku v původním jazyce

We propose a novel Graph Neural Network-based method for segmentation based on data fusion of multimodal Scanning Electron Microscope (SEM) images. In most cases, Backscattered Electron (BSE) images obtained using SEM do not contain sufficient information for mineral segmentation. Therefore, imaging is often complemented with point-wise Energy-Dispersive X-ray Spectroscopy (EDS) spectral measurements that provide highly accurate information about the chemical composition but that are time-consuming to acquire. This motivates the use of sparse spectral data in conjunction with BSE images for mineral segmentation. The unstructured nature of the spectral data makes most traditional image fusion techniques unsuitable for BSE-EDS fusion. We propose using graph neural networks to fuse the two modalities and segment the mineral phases simultaneously. Our results demonstrate that providing EDS data for as few as 1% of BSE pixels produces accurate segmentation, enabling rapid analysis of mineral samples. The proposed data fusion pipeline is versatile and can be adapted to other domains that involve image data and point-wise measurements.

Název v anglickém jazyce

Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks

Popis výsledku anglicky

We propose a novel Graph Neural Network-based method for segmentation based on data fusion of multimodal Scanning Electron Microscope (SEM) images. In most cases, Backscattered Electron (BSE) images obtained using SEM do not contain sufficient information for mineral segmentation. Therefore, imaging is often complemented with point-wise Energy-Dispersive X-ray Spectroscopy (EDS) spectral measurements that provide highly accurate information about the chemical composition but that are time-consuming to acquire. This motivates the use of sparse spectral data in conjunction with BSE images for mineral segmentation. The unstructured nature of the spectral data makes most traditional image fusion techniques unsuitable for BSE-EDS fusion. We propose using graph neural networks to fuse the two modalities and segment the mineral phases simultaneously. Our results demonstrate that providing EDS data for as few as 1% of BSE pixels produces accurate segmentation, enabling rapid analysis of mineral samples. The proposed data fusion pipeline is versatile and can be adapted to other domains that involve image data and point-wise measurements.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Pattern Recognition Letters

ISSN

0167-8655

e-ISSN

1872-7344

Svazek periodika

193

Číslo periodika v rámci svazku

193

Stát vydavatele periodika

NL - Nizozemsko

Počet stran výsledku

7

Strana od-do

79-85

Kód UT WoS článku

001479947700001

EID výsledku v databázi Scopus

2-s2.0-105003299502