Abstract
Reproducible research in pattern recognition can be viewed from a number of angles, including code execution, platforms that promote reproducibility, code sharing, or the release of libraries providing access to relevant algorithms in the corresponding disciplines. In this work, after recalling the motivation and classic definitions of reproducible research, we propose an updated overview of the main platforms that might be used for reproducible research. We then review the different libraries that are commonly used by the pattern recognition, computer vision, imaging and geometry processing communities, and we share our experience of developing a research library. In the third part, new advanced editorial investments will be presented, such as the IPOL journal or other IPOL-inspired new initiatives like OVD-SaaS.
This research was made possible by support from the French National Research Agency, in the framework of the projects WoodSeer, ANR-19-CE10-011, ULTRA-LEARN, ANR-20-CE23-0019, and by the SESAME’s OVD-SaaS project from Région Île de France and BPI France, and Ministry of Science, Technology and Innovation of Colombia (Minciencias), call 885 of 2020.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Data extracted from https://portal.paperswithcode.com on 15 May 2023.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
References
Donoho, D.L., Maleki, A., Ur Rahman, I., Shahram, M., Stodden, V.: Reproducible research in computational harmonic analysis. Comput. Sci. Eng. 11(1), 8–18 (2009)
Colom, M., Kerautret, B., Krähenbühl, A.: An overview of platforms for reproducible research and augmented publications. In: Kerautret, B., Colom, M., Lopresti, D., Monasse, P., Talbot, H. (eds.) RRPR 2018. LNCS, vol. 11455, pp. 25–39. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23987-9_2
Yildiz, B., et al.: ReproducedPapers.org: openly teaching and structuring machine learning reproducibility. In: Kerautret, B., Colom, M., Krähenbühl, A., Lopresti, D., Monasse, P., Talbot, H. (eds.) RRPR 2021. LNCS, vol. 12636, pp. 3–11. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-76423-4_1
Lucic, A., Bleeker, M., Jullien, S., Bhargav, S., de Rijke, M.: Reproducibility as a mechanism for teaching fairness, accountability, confidentiality, and transparency in artificial intelligence (2022)
Rampin, R., Chirigati, F., Steeves, V., Freire, J.: ReproServer: making reproducibility easier and less intensive (2018). https://arxiv.org/abs/1808.01406
Rampin, R., Chirigati, F., Shasha, D., Freire, J., Steeves, V.: ReproZip: the reproducibility packer. J. Open Source Softw. 1(8), 107 (2016)
Šimko, T., Heinrich, L., Hirvonsalo, H., Kousidis, D., Rodríguez, D.: REANA: a system for reusable research data analyses. In: EPJ web of conferences, vol. 214, p. 06034. EDP Sciences (2019)
Bonneel, N., Coeurjolly, D., Digne, J., Mellado, N.: Code replicability in computer graphics. ACM Trans. Graph. 39(4), 93-1 (2020)
Stojnic, R., Taylor, R.: Papers with code-a facebook AI project (2018). https://paperswithcode.com. Accessed 30 Aug 2022
Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools (2000)
McCormick, M., Liu, X., Jomier, J., Marion, C., Ibanez, L.: ITK: enabling reproducible research and open science. Front. Neuroinf. 8, 13 (2014)
Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA). IEEE (2011)
The CGAL Project. CGAL User and Reference Manual. 5.4.1 edition (2022)
Tschumperlé, D.: The CIMG library. In: IPOL 2012 Meeting on Image Processing Libraries, p. 4 (2012)
Geogram: a programming library with geometric algorithms. https://github.com/BrunoLevy/geogram
Roynard, M., Carlinet, E., Géraud, T.: An image processing library in modern C++: getting simplicity and efficiency with generic programming. In: Kerautret, B., Colom, M., Lopresti, D., Monasse, P., Talbot, H. (eds.) RRPR 2018. LNCS, vol. 11455, pp. 121–137. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23987-9_12
Auber, D.: Tulip-a huge graph visualization framework. In: Graph Drawing Software, pp. 105–126. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-642-18638-7_5
Vigra: Vision with generic algorithms. https://ukoethe.github.io/vigra. Accessed May 2022
Dgtal: Digital geometry tools and algorithms library. http://dgtal.org
Moulon, P., Monasse, P., Perrot, R., Marlet, R.: OpenMVG: open multiple view geometry. In: Kerautret, B., Colom, M., Monasse, P. (eds.) RRPR 2016. LNCS, vol. 10214, pp. 60–74. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56414-2_5
Tierny, J., Favelier, G., Levine, J.A., Gueunet, C., Michaux, M.: The topology ToolKit. IEEE Trans. Vis. Comput. Graph. (2017). https://topology-tool-kit.github.io/
Perret, B., Chierchia, G., Cousty, J., Guimarães, S.J. F., Kenmochi, Y., Najman, L.: Higra: hierarchical graph analysis. SoftwareX 10, 1–6 (2019). https://github.com/higra/Higra
Harris, C.R., et al.: Array programming with NumPy. Nature 585(7825), 357–362 (2020)
Lam, S.K., Pitrou, A., Seibert, S.: Numba: a llvm-based python jit compiler. In: Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, pp. 1–6 (2015)
Jakob, W., Rhinelander, J., Moldovan, D.: pybind11—seamless operability between c++11 and python (2016). https://github.com/pybind/pybind11
Nicolaï, A., et al.: The approach to reproducible research of the image processing on line (ipol) journal. Informatio 27(1), 76–112 (2022)
Colom, M., Dagobert, T., Franchis, C.D., Gioi, R.G.V., Hessel, C., Morel, J.M.: Using the ipol journal for online reproducible research in remote sensing. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 13, 6384–6390 (2020)
Baron, A.-F., Boulant, O., Panico, I., Vayatis, N.: A compartmental epidemiological model applied to the Covid-19 epidemic. Image Process. Line 11, 105–119 (2021). https://doi.org/10.5201/ipol.2021.323
Di Cosmo, R., Zacchiroli, S.: Software heritage: why and how to preserve software source code. In: iPRES 2017–14th International Conference on Digital Preservation, pp. 1–10 (2017)
Acknowledgement
The authors would like to thank Burak Yildiz from Delft University of Technology for providing statistics on reproducedpapers.org platform and Dean Pleban from the Dagshub platform for helping and orienting the authors to measure user activity. They also thank the reviewers for their valuable comments and corrections and Bruno Levy for pointing us the usage statistics of the Geogram Library.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Colom, M., Hernández, J.A., Kerautret, B., Perret, B. (2023). Development Efforts for Reproducible Research: Platform, Library and Editorial Investment. In: Kerautret, B., Colom, M., Krähenbühl, A., Lopresti, D., Monasse, P., Perret, B. (eds) Reproducible Research in Pattern Recognition. RRPR 2022. Lecture Notes in Computer Science, vol 14068. Springer, Cham. https://doi.org/10.1007/978-3-031-40773-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-40773-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40772-7
Online ISBN: 978-3-031-40773-4
eBook Packages: Computer ScienceComputer Science (R0)