Automation of cleaning and ensembles for outliers detection in questionnaire data

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61989592%3A15510%2F22%3A73613216" target="_blank" >RIV/61989592:15510/22:73613216 - isvavai.cz</a>
Alternative codes found
RIV/61989100:27240/22:10250037
Result on the web
<a href="https://www.sciencedirect.com/science/article/pii/S0957417422010727" target="_blank" >https://www.sciencedirect.com/science/article/pii/S0957417422010727</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1016/j.eswa.2022.117809" target="_blank" >10.1016/j.eswa.2022.117809</a>

Result language
angličtina
Original language name
Automation of cleaning and ensembles for outliers detection in questionnaire data
Original language description
This article is focused on the automatic detection of the corrupted or inappropriate responses in questionnaire data using unsupervised outliers detection. The questionnaire surveys are often used in psychology research to collect self-report data and their preprocessing takes a lot of manual effort. Unlike with numerical data where the distance-based outliers prevail, the records in questionnaires have to be assessed from various perspectives that do not relate so much. We identify the most frequent types of errors in questionnaires. For each of them, we suggest different outliers detection methods ranking the records with the usage of normalized scores. Considering the similarity between pairs of outlier scores (some are highly uncorrelated), we propose an ensemble based on the union of outliers detected by different methods. Our outlier detection framework consists of some well-known algorithms but we also propose novel approaches addressing the typical issues of questionnaires. The selected methods are based on distance, entropy, and probability. The experimental section describes the process of assembling the methods and selecting their parameters for the final model detecting significant outliers in the real-world HBSC dataset.
Czech name
—
Czech description
—

Type
J<sub>imp</sub> - Article in a specialist periodical, which is included in the Web of Science database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2022
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Similar results(10)