The Influence of Text Pre-processing on Plagiarism Detection

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F09%3A00502242" target="_blank" >RIV/49777513:23520/09:00502242 - isvavai.cz</a>

Result on the web

—

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

The Influence of Text Pre-processing on Plagiarism Detection

Original language description

This paper explores the influence of text pre-processing techniques on plagiarism detection. We examine stop-word removal, lemmatization, number replacement, synonymy recognition, and word generalization. We also look into the influence of punctuation and word-order within N-grams. All these techniques are evaluated according to their impact on F1-measure and speed of execution. Our experiments were performed on a Czech corpus of plagiarized documents about politics. At the end of this paper, we proposewhat we consider to be the best combination of text pre-processing techniques.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

JC - Computer hardware and software

OECD FORD branch

—

Project

<a href="/en/project/2C06009" target="_blank" >2C06009: Complex knowledge base tools for natural language communication with the semantic web</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2009

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

International conference recent advances in natural language processing

ISBN

—

ISSN

1313-8502

e-ISSN

—

Number of pages

5

Pages from-to

—

Publisher name

Incoma Ltd.

Place of publication

Shoumen

Event location

Borovets, Bulgaria

Event date

Sep 16, 2009

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—