Results Reproduction (R-squared)

CISER’s Results Reproduction (R-squared) Service

CISER is offering a service where researchers with “revise-and-resubmit” papers can send their data and code to CISER for curation and appraisal to ensure that published results are replicable; and data and code are well documented, reusable, and preserved for current and future generations of researchers.

Replication of results is a core requirement of the scientific method. Satisfying this requirement becomes increasingly complex when data from disparate sources is integrated and reused. While code used for analysis of data must be verified, it is also imperative that code and processes used to clean, integrate and harmonize data be documented and verified. This can be time-intensive and intimidating, even for individual researchers seeking to openly share their work. Work done by research teams producing multiple versions of source code can make replication considerably more complicated. Below we describe the reasons for the service, and the protocols and tools used by CISER to streamline the experience for researchers who avail of the Reproduction of Results service.

Clients: All Cornell social scientists (Faculty, Staff, and Graduate students)

Reasons for the service
  • More and more journals now require materials for reproducing results (such as data, code, and documentation) to be submitted along with the paper and/or be made available upon request by researchers
  • Funding agencies require grant awardees to share their data
  • To give our researcher confidence about the quality of data, code, and documentation they are sharing to the public. We eliminate the uneasiness that others might find inconsistencies between their published results and the results generated by running their code against their data.
  • To ensure that  data and documentation are independently understandable for informed reuse (Peer, et.al., 2015), and that the code being shared runs and produces outputs that are identical to the paper.
  • For transparency and to provide additional material for re-users to understand exactly what the authors did. This will significantly reduce time and financial resources figuring out how the authors came up with the results.
  • Studies with links to code and data gets cited more (Piwowar et., al, 2007)
  • Study gets scrutinized more, and once it passes scrutiny, the research community validates the results and provides legitimacy to the work done by the authors
  • It furthers science and speeds up the process of knowledge transfer. Others can build on the authors’ work which can lead to more citations.
Workflow

We take you through the steps that require your code, data, and other documentation.  The end result would be a publication-ready and independently-understandable package of replication materials for your study.  CISER also offers an optional curated repository for long-term preservation and availability of your replication materials.