Open Science Practices
Open Science Resources
What is Open Science?
The four fundamental principles/goals of Open Science are:
- Transparency in experimental methodology, observation, and collection of data.
- Public availability and reusability of scientific data.
- Public accessibility and transparency of scientific communication.
- Using web-based tools to facilitate scientific collaboration.
While there is no single doctrine or paper that definitively captures open science, open science can be defined as a set of practices that increase the transparency and accessibility of scientific research. Open science practices include pre-registration of research studies and open research data.
Open Science practices and how they lead to better research
Pre-Registration
In essence, preregistration is the practice of registering, in a time-stamped, locked file in an online repository, the hypotheses, methods, and analyses of a scientific study before you conduct it. Pre-registration not only helps control poor practices such as "p-hacking" and "researcher degrees of freedom", pre-registering also makes for solid science because it helps you think through your study thoroughly from the get go. Tim Ryan summarized this notion as a way of thinking of pre-registration "as a tool that you can use, first, to discipline your thought process: what study do you want to run, and how will the results settle a particular question? And second, use it to build credibility with your readers, most of whom are socialized to be reflexively skeptical."
Pre-registration is not akin to handcuffing you. This is a myth. As Tim Ryan noted, "The purpose of pre-registration is not to censor you or to hide potentially interesting patterns. Rather, it’s to give readers the context they need to know what they can conclude from those patterns." If you decide to deviate from your pre-registration analytic strategy, that's fine—you simply need to state that, add an updated addendum to your pre-registration, or note that this set of analyses is exploratory rather than hypothesized.
Preregistration is on the rise in psychologial science, but we still have a long ways to go.
Open Research Data and Materials
Open Data is a strategy for incorporating research data into the permanent scientific record by releasing it under an Open Access license. Whether data is deposited in a purpose-built repository or published as Supporting Information alongside a research article, Open Data practices ensure that data remains accessible and discoverable. For verification, replication, reuse, and enhanced understanding of research. (See PLOS.)
Participating in open data includes uploading your study's data set (often along with syntax or a descriptive file of how variables were coded) to an open science framework, usually where you have pre-registered your study. Open data also includes sharing any materials that would be needed for someone to replicate your study, such as manipulation videos, participant packages, etc. While your article is under-review at a journal, you can make the pre-registration and data set blind to allow for blind peer review as needed. Datasets should be de-identified before uploading to ensure anonymity of participants.
Access to the data set and these materials allows reviewers to provide fully-informed feedback. When your article is published, access to the data allows for reproducibility, preserves the scientific record, and inspires trust and confidence in your work. An increasing number of academic journals are requiring access to data statements, and many require data to be accessible in an open science online repository.
Open Science Badges
Open Science Badges are an incentive developed by the Center for Open Science for authors who share data or materials and who preregister studies and/or analysis plans. Many journals utilize this system, which enhances credibility of the science. Implementing badges is associated with increasing rate of data sharing (Kidwell et al, 2016), as seeing colleagues practice open science signals that new community norms have arrived.
As described on the APA site and .There are three badge categories:
- Open Data: All data necessary to reproduce the reported results that are digitally shareable are made publicly available. Information necessary for replication (e.g., codebooks or metadata) must be included.
- Open Materials: All materials necessary to reproduce the reported results that are digitally shareable, along with descriptions of non-digital materials necessary for replication, are made publicly available.
-
Preregistered: At least one study’s design has been preregistered with descriptions of (a) the research design and study materials, including the planned sample size; (b) the motivating research question or hypothesis; (c) the outcome variable(s); and (d) the predictor variables, including controls, covariates, and independent variables. Results must be fully disclosed. As long as they are distinguished from other results in the article, results from analyses that were not preregistered may be reported in the article.
A subset of this badge is the Preregistered+Analysis Plan badge: At least one study’s design has been preregistered along with an analysis plan for the research—and results are recorded according to that plan.
Open Science Resources
Check out these resources for further information and to get started!
~ Center for Open Science (COS) blog: Cool articles and recent advances
~ Foster Open Science: Clear explanations, training handbook, further links
~ Open Science Framework (osf): OSF is a free, open platform to support your research and enable collaboration. Discover projects, data, materials, and collaborators on OSF that might be helpful to your own research. [This is where I pre-register my studies. Also has great resources. Give it a whirl! Do a search for a topic and see what comes up.]
~ Journal of Open Psychology Data: publishes peer-reviewed data papers describing psychology datasets with high reuse potential.
