Offline Environment for Focused Crawler Evaluation

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F16%3A00243403" target="_blank" >RIV/68407700:21230/16:00243403 - isvavai.cz</a>
Result on the web
—
DOI - Digital Object Identifier
—

Result language
angličtina
Original language name
Offline Environment for Focused Crawler Evaluation
Original language description
Focused crawling is a method for automatic exploration of the World Wide Web focusing on regions, which are relevant to a given topic. Since focused crawlers are often used to collect data for vertical search engines, they received special attention in research community. Unfortunately, dynamic nature of the internet complicates fair comparisons of focused crawlers, because crawling experiments are not repeatable in the longer term. Researchers try to mitigate the impact of a changing environment by running tested crawlers simultaneously. However, this requirement makes extensive comparisons almost impossible. In this work we try to solve this issue by proposing a system for offline evaluation, which allows developers to consistently test their algorithms on fixed subgraph of the internet. This subgraph consists of 3.5 billion web pages and 128.7 billion links. Our experiments with baseline focused crawlers showed that the performance achieved in the offline environment correlates with the performance achieved online. This suggests that the proposed environment, which allows us to carry out repeatable experiments, can be used for focused crawler comparisons.
Czech name
—
Czech description
—

Publication year
2016
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
Proceedings of the 20th International Scientific Student Conferenece POSTER 2016
ISBN
978-80-01-05950-0
ISSN
—
e-ISSN
—
Number of pages
8
Pages from-to
—
Publisher name
Czech Technical University in Prague
Place of publication
Praha
Event location
Praha
Event date
May 24, 2016
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—

Similar results(10)