2023
Nüst, Daniel; Ostermann, Frank O.; Granell-Canut, Carlos
A peer review process for higher reproducibility of publications in GIScience can also work for Earth System Sciences Proceedings Article
In: European Geosciences Union (EGU) General Assembly 2023, pp. EGU23-15384, Copernicus Publications, 2023.
Abstract | Links | BibTeX | Tags: AGILE, GIScience, Reproducibility, Reproducible research
@inproceedings{nust2023a,
title = {A peer review process for higher reproducibility of publications in GIScience can also work for Earth System Sciences},
author = {Daniel Nüst and Frank O. Ostermann and Carlos Granell-Canut},
doi = {https://doi.org/10.5194/egusphere-egu23-15384},
year = {2023},
date = {2023-04-24},
booktitle = {European Geosciences Union (EGU) General Assembly 2023},
pages = {EGU23-15384},
publisher = {Copernicus Publications},
abstract = {The Reproducible AGILE initiative (https://reproducible-agile.github.io/) successfully established a code execution procedure following the CODECHECK principles (https://doi.org/10.12688/f1000research.51738.2) at the AGILE conference series (https://agile-online.org/conference). The AGILE conference is a medium-sized community-led conference in the domains of Geographic Information Science (GIScience), geoinformatics, and related fields. The conference is organised under the umbrella of the Association of Geographic Information Laboratories in Europe (AGILE).},
keywords = {AGILE, GIScience, Reproducibility, Reproducible research},
pubstate = {published},
tppubtype = {inproceedings}
}
The Reproducible AGILE initiative (https://reproducible-agile.github.io/) successfully established a code execution procedure following the CODECHECK principles (https://doi.org/10.12688/f1000research.51738.2) at the AGILE conference series (https://agile-online.org/conference). The AGILE conference is a medium-sized community-led conference in the domains of Geographic Information Science (GIScience), geoinformatics, and related fields. The conference is organised under the umbrella of the Association of Geographic Information Laboratories in Europe (AGILE).
2021
Ostermann, Frank; Nüst, Daniel; Granell-Canut, Carlos; Hofer, Barbara; Konkol, Markus
Reproducible Research and GIScience: an evaluation using GIScience conference papers Proceedings Article
In: Janowicz, K.; Verstegen, J. A. (Ed.): Proceedings of the 11th International Conference on Geographic Information Science - Part II, pp. 2:1–2:16, LIPIcs, 2021, ISBN: 78-3-95977-208-2.
Abstract | Links | BibTeX | Tags: Geographic Information Systems (GIS), Replicability, Reproducibility
@inproceedings{Ostermann2021a,
title = {Reproducible Research and GIScience: an evaluation using GIScience conference papers},
author = {Frank Ostermann and Daniel Nüst and Carlos Granell-Canut and Barbara Hofer and Markus Konkol },
editor = {K. Janowicz and J.A. Verstegen },
doi = {https://doi.org/10.4230/LIPIcs.GIScience.2021.II.2},
isbn = {78-3-95977-208-2},
year = {2021},
date = {2021-09-30},
booktitle = {Proceedings of the 11th International Conference on Geographic Information Science - Part II},
volume = {208},
pages = {2:1--2:16},
publisher = {LIPIcs},
abstract = {GIScience conference authors and researchers face the same computational reproducibility challenges as authors and researchers from other disciplines who use computers to analyse data. Here, to assess the reproducibility of GIScience research, we apply a rubric for assessing the reproducibility of 75 conference papers published at the GIScience conference series in the years 2012-2018. Since the rubric and process were previously applied to the publications of the AGILE conference series, this paper itself is an attempt to replicate that analysis, however going beyond the previous work by evaluating and discussing proposed measures to improve reproducibility in the specific context of the GIScience conference series. The results of the GIScience paper assessment are in line with previous findings: although descriptions of workflows and the inclusion of the data and software suffice to explain the presented work, in most published papers they do not allow a third party to reproduce the results and findings with a reasonable effort. We summarise and adapt previous recommendations for improving this situation and propose the GIScience community to start a broad discussion on the reusability, quality, and openness of its research. Further, we critically reflect on the process of assessing paper reproducibility, and provide suggestions for improving future assessments. The code and data for this article are published at https://doi.org/10.5281/zenodo.4032875.},
keywords = {Geographic Information Systems (GIS), Replicability, Reproducibility},
pubstate = {published},
tppubtype = {inproceedings}
}
GIScience conference authors and researchers face the same computational reproducibility challenges as authors and researchers from other disciplines who use computers to analyse data. Here, to assess the reproducibility of GIScience research, we apply a rubric for assessing the reproducibility of 75 conference papers published at the GIScience conference series in the years 2012-2018. Since the rubric and process were previously applied to the publications of the AGILE conference series, this paper itself is an attempt to replicate that analysis, however going beyond the previous work by evaluating and discussing proposed measures to improve reproducibility in the specific context of the GIScience conference series. The results of the GIScience paper assessment are in line with previous findings: although descriptions of workflows and the inclusion of the data and software suffice to explain the presented work, in most published papers they do not allow a third party to reproduce the results and findings with a reasonable effort. We summarise and adapt previous recommendations for improving this situation and propose the GIScience community to start a broad discussion on the reusability, quality, and openness of its research. Further, we critically reflect on the process of assessing paper reproducibility, and provide suggestions for improving future assessments. The code and data for this article are published at https://doi.org/10.5281/zenodo.4032875.