An Empirical Comparison of Web Content Extraction Algorithms

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A5SWUZ4MU" target="_blank" >RIV/00216208:11320/23:5SWUZ4MU - isvavai.cz</a>

Výsledek na webu

<a href="https://dl.acm.org/doi/10.1145/3539618.3591920" target="_blank" >https://dl.acm.org/doi/10.1145/3539618.3591920</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3539618.3591920" target="_blank" >10.1145/3539618.3591920</a>

Jazyk výsledku

angličtina

Název v původním jazyce

An Empirical Comparison of Web Content Extraction Algorithms

Popis výsledku v původním jazyce

"Main content extraction from web pages—sometimes also called boilerplate removal—has been a research topic for over two decades. Yet despite web pages being delivered in a machine-readable markup format, extracting the actual content is still a challenge today. Even with the latest HTML5 standard, which defines many semantic elements to mark content areas, web page authors do not always use semantic markup correctly or to its full potential, making it hard for automated systems to extract the relevant information. A high-precision, high-recall content extraction is crucial for downstream applications such as search engines, AI language tools, distraction-free reader modes in users’ browsers, and other general assistive technologies. For such a fundamental task, however, surprisingly few openly available extraction systems or training and benchmarking datasets exist. Even less research has gone into the rigorous evaluation and a true apples-to-apples comparison of the few extraction systems that do exist. To get a better grasp on the current state of the art in the field, we combine and clean eight existing human-labeled web content extraction datasets. On the combined dataset, we evaluate 14 competitive main content extraction systems and five baseline approaches. Finally, we build three ensembles as new state-of-the-art extraction baselines. We find that the performance of existing systems is quite genre-dependent and no single extractor performs best on all types of web pages."

Název v anglickém jazyce

An Empirical Comparison of Web Content Extraction Algorithms

Popis výsledku anglicky

"Main content extraction from web pages—sometimes also called boilerplate removal—has been a research topic for over two decades. Yet despite web pages being delivered in a machine-readable markup format, extracting the actual content is still a challenge today. Even with the latest HTML5 standard, which defines many semantic elements to mark content areas, web page authors do not always use semantic markup correctly or to its full potential, making it hard for automated systems to extract the relevant information. A high-precision, high-recall content extraction is crucial for downstream applications such as search engines, AI language tools, distraction-free reader modes in users’ browsers, and other general assistive technologies. For such a fundamental task, however, surprisingly few openly available extraction systems or training and benchmarking datasets exist. Even less research has gone into the rigorous evaluation and a true apples-to-apples comparison of the few extraction systems that do exist. To get a better grasp on the current state of the art in the field, we combine and clean eight existing human-labeled web content extraction datasets. On the combined dataset, we evaluate 14 competitive main content extraction systems and five baseline approaches. Finally, we build three ensembles as new state-of-the-art extraction baselines. We find that the performance of existing systems is quite genre-dependent and no single extractor performs best on all types of web pages."

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

"Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval"

ISBN

978-1-4503-9408-6

ISSN

—

e-ISSN

—

Počet stran výsledku

10

Strana od-do

2594-2603

Název nakladatele

ACM

Místo vydání

Taipei Taiwan

Místo konání akce

Taipei Taiwan

Datum konání akce

1. 1. 2023

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—