Reproducible AGILE is implements a reproducibility review as part of the AGILE Conference.
Reproducibility Review
The reproducibility review of full paper submissions at AGILE conferences implements the CODECHECK principles for an independent execution of computations underlying research articles as part of the peer review at the AGILE conference. You can find all of AGILE's reproducibility reviews in the CODECHECK Register.
In short, the AGILE realisation of a CODECHECK means that one reproducibility reviewer attempts to reproduce the computations underlying the accepted full papers of the AGILE conference series. If a (partially) successful reproduction is achieved, a reproducibility review is published. The reproducibility review reports on the results and extent of the reproduction and it is published in citable form. All reproduced papers receive the Reproducible AGILE badge in the proceedings list and on the cover page of the article.
Reproducible Paper Guidelines
The following requirement must be complied to by all submissions to the AGILE conference:
- Full and short papers submitted to the AGILE conference have to include a Data and Software Availability section which documents data, software, and computational infrastructure to support reproduction, or mentions reasons for not publishing them.
Non-compliance is grounds for a desk reject.
See the following pages for different perspectives on the guidelines:
- Reproducibility checklist
- Author guidelines for reproducibility
- Reproducibility reviewer guidelines
- Scientific Reviewer Guidelines concerning the reproducibility review
The normative version of the guidelines is published as a single document on OSF: Reproducible Publications at AGILE Conferences – Guidelines for Authors, Scientific Reviewers, and Reproducibility Reviewers 🇺🇸🇬🇧🇳🇿🇦🇺 (December 2020)
Reproducible Publications at AGILE Conferences – Spanish translation 🇪🇸🇲🇽🇦🇷🇨🇴🇪🇨🇧🇴🇨🇷🇨🇺🇩🇴🇸🇻🇬🇶🇬🇹🇭🇳🇳🇮🇵🇦🇵🇾🇵🇪🇵🇷🇺🇾🇻🇪 (version os of July 2019)
The main OSF project with all documentation about the contributors and the genesis of the guidelines can be found at https://osf.io/phmce/.
Reproducibility Reviewers
See the left hand menu for information on each annual reproducibility review and the contributors.
Being a member of the reproducibility committee is an excellent opportunity for early career researchers (ECRs) to build knowledge in scholarly publishing processes and to apply and deepen practical research skills around data and code.
Review Process
TODO: https://docs.google.com/document/d/1JHCQV7GP3YkKwp0Nii3dt3p3Y45hU56Xz2cr-xJVz34/edit?tab=t.0#heading=h.oheeg2s92zdm on a new page
Reproducibility Officer
Daniel Nüst is AGILE's reproducibility officer. He mainly works to ensure continuity of the reproducibility efforts at AGILE and regularly serves as a co-chair of the conference reproducibility committee. Contact Daniel with your questions: