Research Reproducibility 2018: Short Course

Course Instructors

Vicky Steeves

Vicky Steeves

Vicky Steeves is the Librarian for Research Data Management and Reproducibility, a dual appointment between New York University's Division of Libraries and Center for Data Science. Her research centers on integrating reproducible practices into the research workflow, advocating openness in all facets of research, and building/contributing to open infrastructure. She is the co-founder of the LIS Scholarship Archive, an open-source, open access repository for LIS (and allied fields) work. She also works on the ReproZip project, an open source tool that enables full computational reproducibility of research.

Hamid Saimdoust

Hamid Siamdoust

Hamid Siamdoust is a research fellow at University of Utah’s Department of Philosophy. Hamid has worked on issues of research reproducibility since 2008. His other interests include experimental philosophy and moral psychology.  


Subscribe to our mailing list

* indicates required

Short Course: Reproducibility


Monday-Thursday, June 11-14, 2018 10am-4pm & Conference Friday
Location: Carolyn Tanner Irish Humanities Building Auditorium, University of Utah
Cost: FREE; CME available

Registration is Closed

This 5-day course provides an opportunity to explore issues of research reproducibility in-depth in a seminar-type setting, followed by hands-on sessions to learn actionable, practical solutions to make your own work more reproducible. You can choose to attend some or all of the sessions. The final day of the workshop is the conference Building Research Integrity through Reproducibility which requires a separate registration.

Course Overview

Various fields in the natural and social sciences face a ‘crisis of confidence’. Broadly, this crisis amounts to a pervasiveness of non-reproducible results in the published literature. For example, in the field of biomedicine, Amgen published findings that out of 53 landmark published results of pre-clinical studies, only 11% could be replicated successfully. This crisis is not confined to biomedicine. Areas that have recently received attention for non-reproducibility include biomedicine, economics, political science, psychology, as well as philosophy. Some scholars anticipate the expansion of this crisis to other disciplines.

This course explores the state of reproducibility. After giving a brief historical perspective, case studies from different disciplines (biomedicine, psychology, and philosophy) are examined to understand the issues concretely. Subsequently, problems that lead to non-reproducibility are discussed as well as possible solutions and paths forward.

If you are interested in getting academic credit for this course, register here. Course numbers are PHIL 5192/6192-002.

Short Course, Afternoon Sessions Materials can be found here.

Course Outline (PDF link)


Morning Sessions

Instructor: Hamid Siamdoust

Afternoon Sessions

Instructor: Vicky Steeves

June 11

  • Introduction: The Problem of Non-Reproducibility
  • Reproducibility: A Historical Perspective
  • State of Reproducibility: Bad Apples or Systemic Problems

video 1 and video 2

  • Git
  • GitLab/GitHub

video 1 and video 2

June 12

  • Biomedicine
  • Psychology
  • Philosophy

video 1 and video 2

  • OpenRefine
  • R (including RStudio)


June 13

  • Statistical Methods
  • Fraud
  • Incentives, Transparency, and Accountability
  • Jupyter Notebooks
  • ReproZip + VisTrails

June 14

  • Solutions Part I – Various Proposals
  • Solutions – Part II – Replications: Problems and Promise
  • Problems of Implementing Solutions
  • OSF

June 15

   Building Research Integrity through Reproducibility Conference

Continuing Education Credit

ACCME Accredited Provider LogoAccreditation: The University Of Utah School Of Medicine is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.

AMA Credit: The University of Utah School of Medicine designates this live activity for a maximum of 33.00 AMA PRA Category 1 Credit(s)™.  Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Disclosure: None of the faculty or planners or anyone in control of content for this continuing medical education activity have any relevant financial relationships since the content does not cover any products/services of a commercial interest; therefore, there are no relevant financial relationships to disclose.

Nondiscrimination and Disability Accommodation Statement: The University of Utah does not exclude, deny benefits to or otherwise discriminate against any person on the basis of race, color, national origin, sex, disability, age, veteran’s status, religion, gender identity/expression, genetic information, or sexual orientation in admission to or participation in its programs and activities.  Reasonable accommodations will be provided to qualified individuals with disabilities upon request, with reasonable notice.  Requests for accommodations or inquiries or complaints about University nondiscrimination and disability/access policies may be directed to the Director, OEO/AA, Title IX/Section 504/ADA Coordinator, 201 S President’s Circle, RM 135, Salt Lake City, UT 84112, 801-581-8365 (Voice/TTY), 801-585-5746 (Fax).


Research Reproducibility 2018 (Grand Rounds, Short Course, and Conference) is made possible by:

The Office of Research Integrity, The Department of Health and Human Services (ORIIR170034); Vice President for Research Office, University of Utah; Center for Clinical and Translational Science, University of Utah (UL1TR001067); Spencer S. Eccles Health Sciences Library, University of Utah; and Department of Philosophy, University of Utah. Short Course funding is also provided in part through a subaward from the MidContinental Region of the National Network of Libraries of Medicine under cooperative agreement number UG4LM012344.

Office of Research Integrity logo     UHealth    NNLM MCR logo
EHSL Logo     University of Utah Department of Philosophy logo