Heimatkunde: Dataset for Multi-Modal Historical Document Analysis
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F24%3A43971565" target="_blank" >RIV/49777513:23520/24:43971565 - isvavai.cz</a>
Result on the web
<a href="https://www.scitepress.org/Link.aspx?doi=10.5220/0012428500003636" target="_blank" >https://www.scitepress.org/Link.aspx?doi=10.5220/0012428500003636</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.5220/0012428500003636" target="_blank" >10.5220/0012428500003636</a>
Alternative languages
Result language
angličtina
Original language name
Heimatkunde: Dataset for Multi-Modal Historical Document Analysis
Original language description
This paper introduces a novel Heimatkunde dataset comprising printed documents in German, specifically designedfor evaluating layout analysis methods with a focus on multi-modality. The dataset is openly accessiblefor research purposes. The study further presents baseline results for instance segmentation and multi-modalelement classification. Three advanced models, Mask R-CNN, YOLOv8, and LayoutLMv3, are employed forinstance segmentation, while a fusion-based model integrating BERT and various vision Transformers are proposedfor multi-modal classification. Experimental findings reveal that optimal bounding box segmentation isachieved with YOLOv8 using an input image size of 1280 pixels, and the best segmentation mask is producedby LayoutLMv3 with PubLayNet weights. Moreover, the research demonstrates superior multi-modal classificationresults using BERT for textual and Vision Transformer for image modalities. The study concludesby suggesting the integration of the proposed models into the historical Porta fontium portal to enhance theinformation retrieval from historical data.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 16th International Conference on Agents and Artificial Intelligence
ISBN
978-989-758-680-4
ISSN
2184-3589
e-ISSN
2184-433X
Number of pages
7
Pages from-to
995-1001
Publisher name
ScitePress
Place of publication
Setúbal
Event location
Řím, Itálie
Event date
Feb 24, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—