Deciding What to Replicate: A Decision Model for Replication Study Selection Under Resource and Knowledge Constraints

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11230%2F23%3A10435755" target="_blank" >RIV/00216208:11230/23:10435755 - isvavai.cz</a>

Alternative codes found

RIV/00216208:11410/23:10435755 RIV/61384399:31130/23:00057682

Result on the web

<a href="https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=OEao4DXvpc" target="_blank" >https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=OEao4DXvpc</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1037/met0000438" target="_blank" >10.1037/met0000438</a>

Result language

angličtina

Original language name

Deciding What to Replicate: A Decision Model for Replication Study Selection Under Resource and Knowledge Constraints

Original language description

Robust scientific knowledge is contingent upon replication of original findings. However, replicating researchers are constrained by resources, and will almost always have to choose one replication effort to focus on from a set of potential candidates. To select a candidate efficiently in these cases, we need methods for deciding which out of all candidates considered would be the most useful to replicate, given some overall goal researchers wish to achieve. In this article we assume that the overall goal researchers wish to achieve is to maximize the utility gained by conducting the replication study. We then propose a general rule for study selection in replication research based on the replication value of the set of claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by conducting a replication of the claim, and is a function of (a) the value of being certain about the claim, and (b) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value. Our goal in this article is not to define concrete guidelines for study selection, but to provide the necessary theoretical foundations on which such concrete guidelines could be built.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

50101 - Psychology (including human - machine relations)

Project

—

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Psychological Methods

ISSN

1082-989X

e-ISSN

1939-1463

Volume of the periodical

28

Issue of the periodical within the volume

2

Country of publishing house

US - UNITED STATES

Number of pages

14

Pages from-to

438-451

UT code for WoS article

000733130200001

EID of the result in the Scopus database

2-s2.0-85122340446