The GIScience 2020 (-21) conference was held in two years due to pandemics. The second part of the conference also took place virtually at the end of September 2021. We were unable to visit the Adam Mickiewicz University in Poznan, which was the local conference organizer.

The conference was held on a single track, which made it easy for participants to attend all sessions. Indeed, links to all of recording , including two keynotes, are available at https://giscience.org/program/. On the final day, our small team of open science enthusiasts presented our paper entitled “Reproducible Research and GIScience: an evaluation using GIScience conference papers“. It replicated the methodology we used to analyse the level of reproduction of AGILE papers , applied this time to a new data set: GIScience papers published in the 2012-2018 editions.

We do not find notable differences between the two conferences. The good news is that the GIScience papers exposed a minimal level of documentation regarding reproduction. The bad news is that the input data for the analysed papers was often poorly documented, which can definitely pose serious barriers to long-term reproduction.

The abstract of the paper appears below.

GIScience conference authors and researchers face the same computational reproducibility challenges as authors and researchers from other disciplines who use computers to analyse data. Here, to assess the reproducibility of GIScience research, we apply a rubric for assessing the reproducibility of 75 conference papers published at the GIScience conference series in the years 2012-2018. Since the rubric and process were previously applied to the publications of the AGILE conference series, this paper itself is an attempt to replicate that analysis, however going beyond the previous work by evaluating and discussing proposed measures to improve reproducibility in the specific context of the GIScience conference series. The results of the GIScience paper assessment are in line with previous findings: although descriptions of workflows and the inclusion of the data and software suffice to explain the presented work, in most published papers they do not allow a third party to reproduce the results and findings with a reasonable effort. We summarise and adapt previous recommendations for improving this situation and propose the GIScience community to start a broad discussion on the reusability, quality, and openness of its research. Further, we critically reflect on the process of assessing paper reproducibility, and provide suggestions for improving future assessments. The code and data for this article are published at https://doi.org/10.5281/zenodo.4032875.

Cite our paper as

Frank Ostermann, Daniel Nüst, Carlos Granell, Barbara Hofer, Markus Konkol. Reproducible Research and GIScience: an evaluation using GIScience conference papers. K. Janowicz and J.A. Verstegen (Eds.): Proceedings of the 11th International Conference on Geographic Information Science (GIScience 2021) – Part II. LIPIcs, Volume 208, pp 2:1–2:16. ISBN 978-3-95977-208-2. doi:10.4230/LIPIcs.GIScience.2021.II.2.