You are here

Article Text

Article menu

Download PDFPDF

Systematic review
Assessment of transparent and reproducible research practices in the psychiatry literature
  1. Caroline Elizabeth Sherry1,
  2. Jonathan Z Pollard2,
  3. Daniel Tritz1,
  4. Branden K Carr3,
  5. Aaron Pierce3 and
  6. Matt Vassar1
  1. 1Oklahoma State University Center for Health Sciences, Tulsa, Oklahoma, USA
  2. 2University of Missouri Kansas City School of Medicine, Kansas City, Missouri, USA
  3. 3Psychiatry, Oklahoma State University Medical Center, Tulsa, Oklahoma, USA
  1. Correspondence to Ms Caroline Elizabeth Sherry; caroline.sherry{at}okstate.edu

Abstract

Background Reproducibility is a cornerstone of scientific advancement; however, many published works may lack the core components needed for study reproducibility.

Aims In this study, we evaluate the state of transparency and reproducibility in the field of psychiatry using specific indicators as proxies for these practices.

Methods An increasing number of publications have investigated indicators of reproducibility, including research by Harwicke et al, from which we based the methodology for our observational, cross-sectional study. From a random 5-year sample of 300 publications in PubMed-indexed psychiatry journals, two researchers extracted data in a duplicate, blinded fashion using a piloted Google form. The publications were examined for indicators of reproducibility and transparency, which included availability of: materials, data, protocol, analysis script, open-access, conflict of interest, funding and online preregistration.

Results This study ultimately evaluated 296 randomly-selected publications with a 3.20 median impact factor. Only 107 were available online. Most primary authors originated from USA, UK and the Netherlands. The top three publication types were cohort studies, surveys and clinical trials. Regarding indicators of reproducibility, 17 publications gave access to necessary materials, four provided in-depth protocol and one contained raw data required to reproduce the outcomes. One publication offered its analysis script on request; four provided a protocol availability statement. Only 107 publications were publicly available: 13 were registered in online repositories and four, ten and eight publications included their hypothesis, methods and analysis, respectively. Conflict of interest was addressed by 177 and reported by 31 publications. Of 185 publications with a funding statement, 153 publications were funded and 32 were unfunded.

Conclusions Currently, Psychiatry research has significant potential to improve adherence to reproducibility and transparency practices. Thus, this study presents a reference point for the state of reproducibility and transparency in Psychiatry literature. Future assessments are recommended to evaluate and encourage progress.

  • retrospective studies
  • sample size
  • Sampling studies
  • research design
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/gpsych-2019-100149

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Presented at OSU-CHS Research Conference 2020

    • Contributors All authors have contributed substantially to the planning, conduct and reporting of the work described in the article, including, but not limited to; Study design, data acquisition, data analysis, manuscript drafting and final manuscript approval. CES and JZP collaborated on the extraction, validation, organisation, analysis and interpretation of all data. CES was also responsible for team organisation and the manuscript formatting, revision and submission. DT designed methods, compiled the publication list, led data extraction training and assisted with data interpretation and manuscript editing. BKC and AP contributed to data interpretation and writing the introduction and discussion sections. MV provided advisement and leadership in data interpretation, scientific writing and manuscript editing.

    • Funding This study was funded through the 2019 Presidential Research Fellowship Mentor–Mentee Program at Oklahoma State University Center for Health Sciences.

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement All protocols, materials and other pertinent information are available on Open Science Framework (https://osf.io/n4yh5/). Comprehensive results are accessible online in Supplementary Tables A and B.