Reproducibility in Management Science

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14560%2F24%3A00135279" target="_blank" >RIV/00216224:14560/24:00135279 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/61384399:31130/24:00060119

Výsledek na webu

<a href="https://pubsonline.informs.org/doi/full/10.1287/mnsc.2023.03556" target="_blank" >https://pubsonline.informs.org/doi/full/10.1287/mnsc.2023.03556</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1287/mnsc.2023.03556" target="_blank" >10.1287/mnsc.2023.03556</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Reproducibility in Management Science

Popis výsledku v původním jazyce

With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95 % of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29 % of articles at least part of the dataset was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68 %. These figures represent a significant increase compared to the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, out of which 55 % could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness.

Název v anglickém jazyce

Reproducibility in Management Science

Popis výsledku anglicky

With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95 % of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29 % of articles at least part of the dataset was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68 %. These figures represent a significant increase compared to the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, out of which 55 % could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

50200 - Economics and Business

Projekt

—

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

MANAGEMENT SCIENCE

ISSN

0025-1909

e-ISSN

1526-5501

Svazek periodika

70

Číslo periodika v rámci svazku

3

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

14

Strana od-do

1343-1356

Kód UT WoS článku

001132639700001

EID výsledku v databázi Scopus

2-s2.0-85187694519