Research transparency can be achieved by pre-registering studies and other types of qualitative or quantitative research through specification of "the study's motivation, hypothesis, method, materials, sample, and analysis plan... basically specifying everything except the results and the discussion of the results before the study is completed" (Gernsbacher, 2018a). Research transparency can also refer to providing open access to supporting research materials.
Pre-registration creates a permanent research plan that is date- and time-stamped which can be pointed to in order to prove to yourself, and to everyone else, that you actually tested a relationship that you had predicted (Gernsbacher, 2018a). Pre-registration increases the credibility of your research by guarding against HARKing (Hypothesizing After the Results are Known; Kerr, 1987) and fishing, or p-Hacking (i.e., selectively reporting analyses within a study to garner statistically significant results; Simonsohn, Nelson, & Simmons, 2014, p. 670). You may choose to make your pre-registration private until after publication.
Before you pre-register, use the Transparency Tools on this page to make sure you are doing it right the first time around! PROSPERO requires a PRISMA checklist when pre-registering systematic reviews.
This video provides an overview of the AAPOR (American Association for Public Opinion Research) Transparency Initiative’s minimum disclosure (transparency) standards for public opinion research using descriptive, non-structured (qualitative) data, like focus groups, in-depth interviews, case studies, narrative research, and ethnography. The same principles can be applied to other types of qualitative research. Make sure to check your discipline-specific and departmental standards and guidelines for research transparency and reporting.
American Association for Public Opinion Research. (2015, August 12). Transparency Initiative Informational Module 6 [Video file]. Retrieved from https://youtu.be/vsKM2AVnxEI
Studies are not as strong as they claim, often failing reproducibility tests. In fact, 269 co-authors attempted replicating studies found in three prestigious psychology journals and over 1/2 of them failed (Gernsbacher, 2018a). "Most of our common inferential statistics and null hypothesis significance tests . . . rest on the assumption that we have pre-registered, or at the least, pre-specified, our hypotheses and analysis plans," a practice medical researchers have required when conducting clinical trials for the past 20 years (Gernsbacher, 2018a).
Exploratory Research, or unpredicted analyses in either qualitative or quantitative research, can be pre-registered. It must be identified as exploratory in the title or text of the manuscript, clearly distinguishing it from confirmatory research. Pre-registration is a commitment to documenting your predictions or lack thereof, however, if you are worried about being "scooped," you may keep your pre-registration private until after publication (Gernsbacher, 2018a). If you feel restricted by preregistration, identifying your research as exploratory is the perfect solution. In exploratory research, you may preregister a complex and broad qualitative research question instead of a specific, discrete quantitative hypothesis.
Commitments to Open Science are now being found in academic job ads and position descriptions. Including your commitment in cover letters and research statements, and asking recommendation letter writers to include it as well, can really help you stand out from a crowd of applicants (Gernsbacher, 2018a). See examples:
You can start by following Dr. Morton Ann Gernsbacher's (2018a) steps by: 1) Pre-registering Your Research; (2) Making Your Research Materials Open; and (3) Engaging in Open Data & Open Access Publishing.
Now considered best practice, AsPredicted.org and the Center for Open Science allows researchers to document through pre-registration. Systematic Review protocol can be registered in PROSPERO.
The Centre for Open Science maintains a list of journals that accept pre-registered/registered reports for publication. Peer-reviewed twice, they are accepted in advance before results are known, therefore negative or inconclusive results still get published! Since your pre-registration and your manuscript go under peer-review, getting published is not dependent on how flashy your results are--you are committed to a journal, and they are committed to you (Gernsbacher, 2018a).
image: https://cos.io/rr/
Post your experimental research materials on your lab website, the UND Scholarly Commons, and/or on the Open Science Framework (OSF). Dr. Gernsbacher (2018a) posts her materials on her website and in OSF for greater accessibility and permanency as OSF has a mandate to be around at least 50 more years.
Admittedly, opening up one's data to the word can be a bit unnerving, which is why Dr. Gernsbacher (2018a) recommends engaging in a data checking swap with colleagues, who are not involved in that particular study, to determine whether they can replicate your results. It's true that some data should not be made public, however, low risk data that could benefit others, but is otherwise difficult or cumbersome to acquire, should be made open in the spirit of the global scholarly conversation and a broader dissemination of knowledge to the public and other researchers. You may wait to make your data publicly available until your article is published.
Check out Open Data Repositories like the Open Science Framework and Mendeley Data, and our Data Management guide as well as the additional resources on this page.
It is important to de-identify participant data and to tell your research participants that their de-identified data will be made open by putting a notice in the informed consent documentation (Gernsbacher, 2018a). Additionally, you can provide an example of what the data file will look like--Dr. Gernsbacher (2018a) found no difference in the number of participants consenting to the study when implementing these practices. See samples and research by scrolling down this page.
Consider publishing in an Open Access Journal (see DOAJ), or publish your pre-print or post-print article in our institutional repository, UND Scholarly Commons, and/or a pre-print archive like OSF Preprints or arXiv.org (there are also many discipline-specific preprint archives). You can check SHERPA/RoMEO to see if your article's pre-print or post-print can be published.
Make your abstracts informative |
Your reader may only have access to your abstract. Include your (1) primary hypothesis, (2) major methodology, (3) sample size (4) sampled population, (5) main finding, (6) summary statistic including effect size, and (7) key implication. |
Cite responsibly |
Be accurate. There currently exists a high rate of error in citations in research work. Provide hyperlinked DOIs (Digital Object Identifiers) as a practice in accessibility and permanency. |
Document fully |
Make your data analysis, methods, further reading, etc., publicly available through repositories like UND Scholarly Commons, OSF, PubMed Central, or Mendeley Data. |
Document cohesively |
Consider writing a Technical Report including all supporting research documentation with a table of contents instead of a collection of separate files. It can double as an Open Access lab notebook. |
Show your warts |
Reveal how you made things work in less than ideal circumstances (e.g., lost data, excluded participants, experimenter error, etc.). Identify unpredicted results and lack of support for hypotheses. |
Statement of Transparency |
Consider writing one regarding your data collection and other aspects of your research. |
Constraints on Generality Statement |
Consider writing one to document fully the limits of external validity in your research. |
Distinguish exploratory from confirmatory research |
Identify exploratory research in your manuscript title (e.g., "An Exploratory Study") or in related sections of text (e.g., methodology). |
Check out the many helpful tools and resources for making your research transparent and open that are linked on this page. You may also want to visit our Open Access and Publishing Strategies research guides.
Keynote Address by Dr. Morton Ann Gernsbacher
Research Transparency in Psychological Science: How & Why?
Research Transparency in Psychological Science: How and Why? from University of North Dakota on Vimeo.
The active ingredient of any science, including psychological science, is that its results reproduce. To ensure greater reproducibility, psychological scientists are increasingly taking steps toward greater research transparency. Such steps include preregistering their studies’ goals and analysis plans; making their studies’ research materials available to everyone; making their studies’ data available to everyone; and making their resulting research reports available to everyone. In this presentation, I’ll discuss how everyone can take these steps toward greater research transparency – and why. -- https://arts-sciences.und.edu/psychology/northern_lights/keynote-speaker.cfm