Science, Vol.349, No.6251, ---, 2015
Estimating the reproducibility of psychological science
Aarts AA ,
Anderson JE ,
Anderson CJ ,
Attridge PR ,
Attwood A ,
Axt J ,
Babel M ,
Bahnik S ,
Baranski E ,
Barnett-Cowan M ,
Bartmess E ,
Beer J ,
Bell R ,
Bentley H ,
Beyan L ,
Binion G ,
Borsboom D ,
Bosch A ,
Bosco FA ,
Bowman SD ,
Brandt MJ ,
Braswell E ,
Brohmer H ,
Brown BT ,
Brown K ,
Bruning J ,
Calhoun-Sauls A ,
Callahan SP ,
Chagnon E ,
Chandler J ,
Chartier CR ,
Cheung F ,
Christopherson CD ,
Cillessen L ,
Clay R ,
Cleary H ,
Cloud MD ,
Cohn M ,
Cohoon J ,
Columbus S ,
Cordes A ,
Costantini G ,
Alvarez LDC ,
Cremata E ,
Crusius J ,
DeCoster J ,
DeGaetano MA ,
Della Penna N ,
den Bezemer B ,
Deserno MK ,
Devitt O ,
Dewitte L ,
Dobolyi DG ,
Dodson GT ,
Donnellan MB ,
Donohue R ,
Dore RA ,
Dorrough A ,
Dreber A ,
Dugas M ,
Dunn EW ,
Easey K ,
Eboigbe S ,
Eggleston C ,
Embley J ,
Epskamp S ,
Errington TM ,
Estel V ,
Farach FJ ,
Feather J ,
Fedor A ,
Fernandez-Castilla B ,
Fiedler S ,
Field JG ,
Fitneva SA ,
Flagan T ,
Forest AL ,
Forsell E ,
Foster JD ,
Frank MC ,
Frazier RS ,
Fuchs H ,
Gable P ,
Galak J ,
Galliani EM ,
Gampa A ,
Garcia S ,
Gazarian D ,
Gilbert E ,
Giner-Sorolla R ,
Glockner A ,
Goellner L ,
Goh JX ,
Goldberg R ,
Goodbourn PT ,
Gordon-McKeon S ,
Gorges B ,
Gorges J ,
Goss J ,
Graham J ,
Grange JA ,
Gray J ,
Hartgerink C ,
Hartshorne J ,
Hasselman F ,
Hayes T ,
Heikensten E ,
Henninger F ,
Hodsoll J ,
Holubar T ,
Hoogendoorn G ,
Humphries DJ ,
Hung COY ,
Immelman N ,
Irsik VC ,
Jahn G ,
Jakel F ,
Jekel M ,
Johannesson M ,
Johnson LG ,
Johnson DJ ,
Johnson KM ,
Johnston WJ ,
Jonas K ,
Joy-Gaba JA ,
Kappes HB ,
Kelso K ,
Kidwell MC ,
Kim SK ,
Kirkhart M ,
Kleinberg B ,
Knezevic G ,
Kolorz FM ,
Kossakowski JJ ,
Krause RW ,
Krijnen J ,
Kuhlmann T ,
Kunkels YK ,
Kyc MM ,
Lai CK ,
Laique A ,
Lakens D ,
Lane KA ,
Lassetter B ,
Lazarevic LB ,
LeBel EP ,
Lee KJ ,
Lee M ,
Lemm K ,
Levitan CA ,
Lewis M ,
Lin L ,
Lin S ,
Lippold M ,
Loureiro D ,
Luteijn I ,
Mackinnon S ,
Mainard HN ,
Marigold DC ,
Martin DP ,
Martinez T ,
Masicampo EJ ,
Matacotta J ,
Mathur M ,
May M ,
Mechin N ,
Mehta P ,
Meixner J ,
Melinger A ,
Miller JK ,
Miller M ,
Moore K ,
Moschl M ,
Motyl M ,
Muller SM ,
Munafo M ,
Neijenhuijs KI ,
Nervi T ,
Nicolas G ,
Nilsonne G ,
Nosek BA ,
Nuijten MB ,
Olsson C ,
Osborne C ,
Ostkamp L ,
Pavel M ,
Penton-Voak IS ,
Perna O ,
Pernet C ,
Perugini M ,
Pipitone RN ,
Pitts M ,
Plessow F ,
Prenoveau JM ,
Rahal RM ,
Ratliff KA ,
Reinhard D ,
Renkewitz F ,
Ricker AA ,
Rigney A ,
Rivers AM ,
Roebke M ,
Rutchick AM ,
Ryan RS ,
Sahin O ,
Saide A ,
Sandstrom GM ,
Santos D ,
Saxe R ,
Schlegelmilch R ,
Schmidt K ,
Scholz S ,
Seibel L ,
Selterman DF ,
Shaki S ,
Simpson WB ,
Sinclair HC ,
Skorinko JLM ,
Slowik A ,
Snyder JS ,
Soderberg C ,
Sonnleitner C ,
Spencer N ,
Spies JR ,
Steegen S ,
Stieger S ,
Strohminger N ,
Sullivan GB ,
Talhelm T ,
Tapia M ,
te Dorsthorst A ,
Thomae M ,
Thomas SL ,
Tio P ,
Traets F ,
Tsang S ,
Tuerlinckx F ,
Turchan P ,
Valasek M ,
van 't Veer AE ,
Van Aert R ,
van Assen M ,
van Bork R ,
van de Ven M ,
van den Bergh D ,
van der Hulst M ,
van Dooren R ,
van Doorn J ,
van Renswoude DR ,
van Rijn H ,
Vanpaemel W ,
Echeverria AV ,
Vazquez M ,
Velez N ,
Vermue M ,
Verschoor M ,
Vianello M ,
Voracek M ,
Vuu G ,
Wagenmakers EJ ,
Weerdmeester J ,
Welsh A ,
Westgate EC ,
Wissink J ,
Wood M ,
Woods A ,
Wright E ,
Wu S ,
Zeelenberg M ,
Zuni K
Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
Please enable JavaScript to view the comments powered by Disqus.
