In order to draw causal conclusions about the reasons for differing study results, it is necessary to precisely define the effects under investigation, exclude confounding factors by design, and collect additional measures that make differences between studies identifiable and controllable.
The literature review on the design of replication studies shows that, to date, procedural aspects—such as methods, procedures, and analysis techniques—have been controlled for, while other study characteristics, such as the population studied or the setting, have been neglected.
In empirical study series, various replication factors were therefore specifically varied between studies and unintended differences in study characteristics were controlled for. In particular, the composition of the sample proved to be a key factor influencing the variation in effects, while characteristics such as recruitment time or technical equipment had no substantial influence.
New analysis methods were developed for the statistical control of study differences, which preserve intended variations and adjust for unintended variations. In addition to the theoretical assumptions and estimation procedures, sensitivity analyses are provided and the implementation is illustrated using empirical studies.