Cognitive modelling: Blessing or curse for replication in Psychology?
Psychological science is currently facing a crisis of credibility because researchers have realized that numerous influential psychological studies cannot be replicated. A potential explanation for replication failures is that psychological theories are often underspecified. As a countermeasure against weak theories, it has bees been recommended that psychological studies should make more wide-spread use of formal cognitive modelling to generate more precise predictions. However, it has never been empirically investigated if cognitive modelling analysis is in fact beneficial for replicability.
Given the large number of arbitrary analysis decisions required for cognitive modelling analyses, there is the possibility that cognitive modelling is in fact counterproductive for replicability of psychological findings. In our project, we aim to investigate the replicability of cognitive models based on Bayesian Brain Theory in three exemplary studies. First, we will investigate the reproducibility of the analyses conducted by the authors of the original studies using the original data sets. Second, we will examine the robustness of cognitive modelling analyses by systematically assessing the impact of a variety of theoretically equivalent analysis decisions onto the results. Finally, we will test if we obtain equivalent results as reported in the original studies when we perform exact replication studies of the original experiments.