Open Science Practices

Transparency Guidelines

We take good, transparent, reproducible, and open science very seriously. This means that all published papers will have underwent screening regarding to what extent they have fulfilled Transparency and Openness Promotion (TOP) Guidelines. The TOP Guidelines outline eight modular standards that can be implemented at three different levels (with Level I being the least, and Level III the most rigorous option):

The guidelines summarized above concern any paper published in QCMB that presents empirical data (whether quantitive or qualitative). Data, code, materials, and other documents should be safely and openly stored at the PsychArchives repository. Upon submission of a paper, authors will have to confirm that they have included appropriate statements (e.g., about the availability of data, code, materials, other documents, preregistrations) within their papers, and all papers will be checked regarding their adherence to the journal's adopted TOP standards outlined above. Authors should thus take great care to ensure that their papers meet those standards. Papers that do not meet the standards cannot be published in this journal.

Implementation Guidelines for Authors

1. Citation Standards

All data, program code and other methods should be appropriately cited. Such materials should be recognized as original intellectual contributions and afforded recognition through citation. Articles will not be published until the citations conform to these standards.

1. All data sets and program code used in a publication should be cited in the text and listed in the reference section.
2. References for data sets and program code should include a persistent identifier, such as a Digital Object Identifier (DOI). Persistent identifiers ensure future access to unique published digital objects, such as a text or data set. Persistent identifiers are assigned to data sets by digital archives, such as institutional repositories and partners in the Data Preservation Alliance for the Social Sciences (Data-PASS).
3. Data set citation example: Campbell, Angus, and Robert L. Kahn. American National Election Study, 1948. ICPSR07218v3. Ann Arbor, MI: Interuniversity Consortium for Political and Social Research [distributor], 1999. http://doi.org/10.3886/ICPSR07218.v3

2. Data Transparency

It is our policy to ony publish papers if the data are clearly and precisely documented and are maximally available to any researcher for purposes of reproducing the results or replicating the procedure.

1. Authors reusing data available from public repositories must provide program code, scripts for statistical packages, and other documentation sufficient to allow an informed researcher to precisely reproduce all published results.
2. Authors using original data must: 
   1. make the data available at a trusted digital repository (Note: If all data required to reproduce the reported analyses appears in the article text, tables, and figures then it does not also need to be posted to a repository.)
   2. include all variables, treatment conditions, and observations described in the manuscript
   3. provide a full account of the procedures used to collect, preprocess, clean, or generate the data
3. In rare cases, despite authors’ best efforts, some or all data cannot be shared for legal or ethical reasons. In such cases, authors must inform the editors at the time of submission. This will be taken into account during the review process. Authors are encouraged to anticipate data sharing at the beginning of their projects to provide for these circumstances. It is understood that in some cases access will be provided under restrictions to protect confidential or proprietary information. Editors may grant exceptions to data and material access requirements provided authors:
   1. explain the restrictions on the dataset and how they preclude public access.
   2. provide a public description of the steps others should follow to request access to the data.
   3. provide access to all data for which the constraints do not apply.
4. Data should be made available through PsychArchives (trusted digital repository). Trusted repositories adhere to policies that make data discoverable, accessible, usable, and preserved for the long term. Trusted repositories also assign unique and persistent identifiers. For example these services are offered by partners in the Data Preservation Alliance for the Social Sciences (Data-PASS) and most institutional repositories. Author maintained websites are not compliant with this requirement.
   1. Dissemination of these data may be delayed until publication. Under exceptional circumstances, editors may grant an embargo of the public release of data for at most one year after publication.
   2. Articles accepted for publication will not be assigned a publication date until the above conditions have been met. Authors are responsible for ensuring that their articles continue to meet these conditions. Failure to do so may lead to an editorial expression of concern or retraction of the article.

3. Code Transparency

Only papers whose methods used in the analysis are clearly and precisely documented and are maximally available to any researcher for purposes of reproducing the results or replicating the procedure will be publshed in this journal.

1. Authors reusing data available from public repositories must provide program code, scripts for statistical packages, and other documentation sufficient to allow an informed researcher to precisely reproduce all published results.
2. Authors using original data must: 
   1. make the data available at a trusted digital repository (Note: If all data required to reproduce the reported analyses appears in the article text, tables, and figures then it does not also need to be posted to a repository.)
   2. include all variables, treatment conditions, and observations described in the manuscript
   3. provide a full account of the procedures used to collect, preprocess, clean, or generate the data
3. In rare cases, despite authors’ best efforts, some or all materials cannot be shared for legal or ethical reasons. In such cases, authors must inform the editors at the time of submission. This will be taken into account during the review process. Authors are encouraged to anticipate material sharing at the beginning of their projects to provide for these circumstances. It is understood that in some cases access will be provided under restrictions to protect confidential or proprietary information. Editors may grant exceptions to data and material access requirements provided authors:
   1. explain the restrictions on the dataset and how they preclude public access.
   2. provide a public description of the steps others should follow to request access to the data.
   3. provide access to all data for which the constraints do not apply.
4. Program code should be made available through PsychArchives (trusted digital repository). Trusted repositories adhere to policies that make data discoverable, accessible, usable, and preserved for the long term. Trusted repositories also assign unique and persistent identifiers. For example these services are offered by partners in the Data Preservation Alliance for the Social Sciences (Data-PASS) and most institutional repositories. Author maintained websites are not compliant with this requirement.
   1. Dissemination of these data may be delayed until publication. Under exceptional circumstances, editors may grant an embargo of the public release of data for at most one year after publication.
   2. Articles accepted for publication will not be assigned a publication date until the above conditions have been met. Authors are responsible for ensuring that their articles continue to meet these conditions. Failure to do so may lead to an editorial expression of concern or retraction of the article.

4. Materials Transparency

For articles in the area of the Journal (quantitative psychology), this standard is usually not applicable. In exceptional cases the defined level applies.

It is our policy to publish papers where authors indicate whether the materials used to conduct the research will be made available to any researcher for purposes of reproducing the results or replicating the procedure.

Authors must, in acknowledgments or the first footnote, indicate if they will or will not make their study materials available to other researchers. If an author agrees to make materials available, the author must specify where that material will be available.

5. Design Transparency

For articles in the area of the Journal (quantitative psychology), this standard is usually not applicable. In exceptional cases the defined level applies.

We publish papers where authors follow standards for disclosing key aspects of the research design and data analysis. Authors are required to review the standards available for many research applications from equator-network.org and use those that are relevant for the reported research applications. At manuscript submission, authors must confirm that they reviewed the standards, report whether any standards were relevant for the research application, and confirm that they followed those standards in the manuscript.

6./7. Study and Analysis Preregistration

For articles in the area of the Journal (quantitative psychology), this standard is usually not applicable. In exceptional cases the defined level applies.

We publish papers where authors indicate whether or not the conducted research was preregistered with an analysis plan in an independent, institutional registry (e.g., aspredicted.org, clinicaltrials.gov, socialscienceregistry.org, osf.io, ridie.3ieimpact.org ). Preregistration of studies involves registering the study design, variables, and treatment conditions. Including an analysis plan involves specification of sequence of analyses or the statistical model that will be reported. 

1. Authors must, in acknowledgments or the first footnote, indicate if they did or did not preregister the research with or without an analysis plan in an independent, institutional registry.
2. If an author did preregister the research with an analysis plan, the author must: 
   1. confirm in the text that the study was registered prior to conducting the research with links to the timestamped preregistrations at the institutional registry, and that the preregistration adheres to the disclosure requirements of the institutional registry or those required for the preregistered badge with analysis plans maintained by the Center for Open Science.
   2. Report all preregistered analyses in the text, or, if there were changes in the analysis plan following preregistration, those changes must be disclosed with explanation for the changes.
   3. Clearly distinguish in text analyses that were preregistered from those that were not, such as having separate sections in the results for confirmatory and exploratory analyses.

8. Replication

We encourage submission of replication studies, particularly of research published in this journal. Replication studies are normally reviewed in two stages, the first being results-blind.

1. On initial submission, authors should:

   a. Denote in the cover letter that the manuscript is a replication study submission
   b. Include a full manuscript for the abstract, introduction, and methods without the results and discussion sections. The submitted manuscript must not indicate information about the outcome-relevant results.
   c. The methods must contain a complete analysis plan of what is to be included in the full article.
   d. If relevant, outcome-irrelevant results can be reported to demonstrate, for example, that that experimental manipulations were effective, or outcome variables were measured reliably and conformed to distributional assumptions.

2. If the submission passes initial review, then the authors will submit a complete manuscript for second stage review to confirm that the final report adequately addresses reviewer concerns from the initial submission.