1.1 Current Situation of Research Software

Software creation, extension and reuse is an integral part of research activity. A customised software solution is often necessary to analyse, select and visualise research data, due to the unique nature of each research project. The resulting software can be shared in connection with the research data, or stand on its own as an independent result. Research software is mainly developed by scientists with basic programming skills. The correctness of the code is guaranteed as far as the software works, but the adoption of good practises concerning comments, indentation, documentation, naming, versioning and citation is often rudimentary. Other aspects that are not always considered are the use of resources, referencing and licence-conform reuse of existing software, long-term preservation and responsibilities. Thus, the aim of this text is to show the possibilities for software management plans (SMPs) and to enable infrastructure providers, e.g., libraries and data centers, to quickly get started in the topic.

There is a lot of community-driven work to foster good programming practises among scientists. This is not a contemporary phenomenon, but has instead been true since the use of code in a scientific context. However, this has been gaining increasing attention in recent years due to activities in research data management and the enhanced publication of dataset and code. This awareness building is supported by several initiatives and takes advantage of many instruments that are already commonly used among professional programmers. Among these instruments are the broad adoption of versioning platforms¹; the definition of software metadata schemas² or special citation formats for research software like .cff (Druskat et al., 2021); the introduction of development environments (IDEs) allowing bug fixing, testing and bench-marking; the diffusion of software packages for the half-automatic generation of documentation and reproducible examples.³ Information exchange via language-specific and generic fora⁴ additionally helps with the development of software in research.

The structure of this article leads from the abstract to the concrete. The introduction presents the framework for the present work like motivation, definition and ongoing initiatives around SMPs. This is followed by section 2, which presents and compares five selected SMP templates which are the object of study. Part 3 shows some forms of support in the preparation of a SMP, including five Data Management Plan (DMP) tools. Next, chapter 4 gathers some consideration about SMPs, metadata, FAIR principles and machine-actionability. Section 5 discusses advantages and disadvantages of SMPs also in relation to possible use cases. The article closes with a conclusion as wrap-up to indicate the need for future work.

1.2 Environment for SMPs

Together with the aforementioned applications, tools and procedures for research software, amateur programmers need an instrument to collect ideas and information necessary for their programming activity, which helps them gain awareness and provide for the necessary resources. Such an instrument can be an SMP, which aims to collect all information relevant for the software development in a scientific context to support a structured approach or help distributed teams to adhere to (self-) defined standards.

Albeit no funding organization explicitly requires a software management plan at the moment, there are developments in this direction, as the German Volkswagen Foundation addresses in such a way research software in its Open Science Policy (Volkwagen-Stiftung, 2022). It can be expected that other funding agencies, such as the European Union, will follow. In these cases, it is helpful for service providers such as university libraries, data centers or scientific computing centers to be prepared.

In this article, we aim to collect all relevant information and helpful material to enable end-users to create a SMP that is tailored to the needs of their community and institution. The following text will analyse and highlight some of the advantages of SMPs from various perspectives.

1.3 SMP definition

A software management plan is a living document which supports software development, addressing most relevant aspects. There are different SMP definitions with different focuses and they all have their strengths (see, for example, Martinez-Ortiz et al., 2023 or Alves et al., 2021). The following text is based on a definition that, in addition to a general formulation, attempts to understand an SMP and its possible uses via a descriptive listing. It was developed by a joint working group⁵ of German Initiative for Network Information (DINI) and German competence network for digital long-term storage (nestor) the authors belong to, and is available in German and English at the German national information website on research data management⁶:

A Software Management Plan (SMP) contains general and technical information about the software project, information on quality assurance, release and public availability as well as legal and ethical aspects that affect the software.

The SMP summarises information that adequately describes and documents the creation, documentation, storage, versioning, licensing, archiving and/or publication of the software created or used in a project. Associated hardware and other necessary resources, as well as other associated software and software libraries, text and data publications must also be described and are a special feature of the SMP.

The purpose of an SMP is initially to support the traceability and, if necessary, the long-term usability of the software (for direct application as well as for further processing) and to facilitate user support in the event of queries. The SMP therefore also serves the purpose of quality assurance (see FAIR for Research Software.)

The SMP can be linked to one or more DMPs if the software is used for data generation or processing. SMP and DMP can be summarised as output plans.

This definition is one of the existing proposals for labelling the subject area and will be applied in the following sections. Ultimately, these ideas need to be filled with life. This requires researchers and support personnel who implement these developments in practise and at the same time deal with SMPs on a theoretical level. The following chapters also deal with this transition from theoretical considerations to practise.

1.4 Current initiatives dealing with SMPs

At the international, European and German levels, there are various institutions, working groups and initiatives that are already working on SMPs. The most important players are briefly named and presented below, without any claim for completeness.

2.1 Facilitation of SMP creation through templates

Different approaches can be adopted for writing SMPs, similarly to DMPs. A plan can be prepared from scratch as a result of a brainstorming, or by answering a prepared collection of questions, which is usually provided as a simple text document. Some selected templates are presented and discussed in this chapter, see Figure 1. Meanwhile, templates for management plans that are prepared in special software environments are more common. Such tools enable the provision of different question complexes in a standardised way, along with support in the form of help texts and automated exporting possibilities, e.g., as PDF or via API calls among others. Some selected DMP/SMP tools are presented in the following chapter.

2.2 Overview of the considered SMP templates

For this study, we collected five templates which are available online (see also Figure 1). These are briefly described below in order to show their context of origin, their strengths and possible areas of application.

2.3 Comparison of SMP templates

The five chosen SMP templates are compared in the two following Tables 1 and 2. They differ significantly in coverage, focus, and granularity. All have in common that they act as standalone documents, i.e., they are not dependent from or referencing a DMP.¹³

The longest SMP template is the one from PRESOFT, with just under a hundred questions; the SSI checklist is the shortest with ‘only’ 38 questions. It is striking that each of the templates has a different focus and orientation. Table 1 demonstrates this impressively. Some of the templates come more from project management, others are more closely aligned with DMPs.

The SMPs from ZIB and ELIXIR are closely oriented to software development. Both, therefore, have comparatively many questions in the area of documentation and preservation. The SSI checklist, on the other hand, is aimed primarily at future re-use and less at the phase during software development. One special feature of ELIXIR is the high degree of machine-actionablilty. It is noticeable here that many question sets do not appear that are included in the other SMP templates in different ways. This is certainly also due to the fact that the SSI-SMP template is the oldest and is therefore regarded by many as a benchmark. When using PRESOFT, a particularly significant level of information is collected in the administrative area and in the software description. The MPDL template is again aimed primarily at scientists as self-taught software developers. The width of questions and subject areas is more striking here than the depth to which different aspects are queried. The detailed overview in Table 2 also shows which sub-items are addressed in the SMP template and to what extent.

The two tables help to get a first impression and comparison of the existing SMP templates. The questions of all considered templates have been firstly annotated with the most relevant keyword contained and subsequently clustered heuristically into major categories. Finally, the relative importance of each topic cluster in each catalogue was reported as relative fraction of the number of questions. The obtained structure helps comparing, qualitatively, the question collections. The two tables help to get an initial visual impression of which topics are covered by the various SMP templates and to what extend.¹⁴

The FAIR4RS principles are a significant point of reference. Depending on the internal logic of the SMP templates, they are directly invoked. This is naturally less the case with SMP templates that were created before 2021, although the Findable, Accessible, Interoperable and Reusable (FAIR) Data principles already appear here indirectly as well. However, it is clear that these SMP templates will also be adapted, expanded, readjusted or phased out over time as the needs, assumptions and expectations regarding the use of research software change. This is also reflected in the SMP templates.

3.1 Consulting and training on SMPs

The use of SMPs is still under discussion. Therefore, a comparatively small amount of experience can be drawn on for training and consulting. As a consequence, it should not be surprising if only a few scientists and stakeholders have started to recognise the validity of their use for project and quality management in the development of research software within the own institution.

The increasing attention around research software at scientific institutions will most likely be accompanied by funders paying more attention to the topic. At the same time, more consulting and training offers will also be necessary for users. This can, for example, come from the implementation of the FAIR4RS Principles (Chue Hong et al., 2022) by scientific committees or funding bodies such as the European Commission and the upcoming 10^th EU Framework Programme for Research and Innovation. In the German context, the Helmholtz Association has made significant progress: From 2025 on, all Helmholtz Centres will be able to present research software management procedures in publicly available policies.¹⁵ In addition, an incentive for the publication of research software is being developed (Helmholtz Association, 2022, p. 5). Such institutional incentives can again lead to greater demand for research software management. SMPs can provide support here.

Such SMPs as a service can be offered, for example, by research data managers, information specialists or software experts. For such a service, it is necessary that these professionals are prepared in using SMPs. The sub-working group Training/Further education¹⁶ of the already mentioned DINI/nestor-Working Group Research Data has prepared generic training materials for teaching SMPs that can be used in workshops for this target group (Biernacka and Helbig, 2023. This is a module of Biernacka et al., 2023).

Biernacka and Schulz, 2022, on the other hand, demonstrate how SMPs can be incorporated in Computer Science Higher Education training, providing working materials and use cases for students and researchers (Biernacka and Schulz, 2022). In the long term, it is therefore also recommended that consulting and training on SMPs will be included in the standard study curricula in the area of research software, and also in research data management Biernacka and Helbig, 2023).

Still, most of the existing training materials and documents so far are strongly related to the platform used. This will become visible in 3.3.

3.2 Manuals and guides for SMPs

There are already some handouts about the preparation and use of SMPs. These can be particularly helpful for the use of SMPs as a service. The first generic reference was published by Neil Chue Hong (2014).¹⁷ Since 2018, the German Aerospace Center (DLR) guidelines have also included concrete recommendations for the general use of research software. The guidelines introduces 4 application classes (ac) for scientific software, ranging from small scripts intended for personal use (ac0) to production type software which is intended for use in mission critical systems (ac3). This categorisation of the own software according to scope and area of use can be a substantial help during the development process. The four classes are helpful as an initial orientation for an assessment of the code. Is it a small script or an entire infrastructure? Depending on the selection, fewer or more measures are necessary (Schlauch et al., 2018). For life sciences, recommendations for a low-threshold SMP have also already been developed and discussed in the context of ELIXIR in 2021 (Alves et al., 2021).¹⁸ The most recent contribution is the practical guide by the Netherlands eScience Center and the Dutch Research Council (NWO) from 2022 (Martinez-Ortiz et al., 2023). However, the ongoing discussions around SMPs have put SMPs in the mainstream in 2023, so that further handouts, open training material and more can be expected soon (see chapter 1.4 for current initiatives).

3.3 Implementation in DMP tools

Currently, there is no service that offers SMPs alone. In all cases, SMPs are offered as part of a platform that was actually designed for DMPs. Not surprisingly, these are always community-driven platforms from a scientific context that are committed to the open source idea. The platforms and their content have evolved. At the same time, it should be positively emphasised that for new services – such as an SMP offer – it is not always necessary to set up new infrastructure. Rather, it can make sense to expand existing solutions in order to make them usable for new demands.

4.1 FAIR4RS principles, validity, and limitations

The FAIR Data principles, which constitute a set of guidelines facilitating sharing and re-use of data, were officially published in Scientific Data in 2016 by FORCE11 (Wilkinson et al., 2016), a consortium consisting of researchers, librarians, archivists, publishers and research funding bodies. The primary objective underpinning these principles revolves around ensuring data to be FAIR effectively by both humans and machines. Despite their non-standard status, these principles have garnered growing significance due to their compelling advocacy for data reusability. Attaining this objective demands incorporating the principles throughout the entire spectrum of Research Data Management (RDM) (Engelhardt et al., 2022).

It soon became clear that the FAIR principles needed to be adapted and modified to include research software. This is also because software, despite being closely related to research data, has special characteristics. Among other things, it is characterised by the fact that it is executable, that its versions are frequently updated and that it is rarely created from scratch. Recognising this need, Lamprecht et al. reformulated the FAIR principles in a publication on the progress of the joint working group, tailoring them to be more aptly applied to software (Lamprecht et al., 2020). Subsequently, in 2022, these revised FAIR4RS principles were officially published by Chue Hong at al. as a direct outcome of the collaborative efforts of the FAIR for Research Software Working Group (FAIR4RS WG) within the RDA (Chue Hong et al., 2022).

There are many obvious benefits of FAIR4RS, such as improving the discoverability of research software, enabling its identification and reuse by other researchers and communities, and facilitating compatibility and integration with different tools and platforms. However, certain barriers hinder their successful implementation. Among the significant limitations is the inherent complexity of research software, which makes it challenging to apply FAIR4RS principles uniformly across all software types. Furthermore, implementing FAIR principles for research software may require significant resources, including time, expertise and infrastructure. Another challenge is to convey the benefits of FAIR4RS to developers, funded projects and the wider research community (Castro et al., 2021). While the FAIR principles have become broadly adopted and are viewed as a benchmark for research data management, the FAIR4RS principles remain relatively unrecognised or not widely embraced, particularly as RSEs ‘typically’ do not pursue conventional academic career pathways and may not place significant emphasis on traditional metrics like citations or recognition. Similar to the implementation of the FAIR principles, it lends itself to integrate the FAIR4RS principles into the SMP.

4.2 Metadata and research software

As for any other publicly available digital object, research software needs to be described with structured and machine-interpretable metadata in order to achieve interoperability of SMPs within the software development process and to make it easier for researchers to discover, understand and reuse it, and to conform to the FAIR4RS principles. In the realm of research software, several initiatives have been developed to address the need for machine-interoperable metadata, simplifying the process of describing and sharing software resources effectively. Notable among these are Bioschemas and CodeMeta. Bioschemas is a toolbox to add structured metadata to research outputs, including software, with a particular focus on life sciences. It defines a set of metadata schemas and vocabularies, built on top of existing technologies and standards, that can be used to represent such tools in Web pages and applications, and provides tools such as the ComputationalTool and the Markup Generator (bioschemas.org, 2022). CodeMeta is a minimal metadata schema for scientific software and code, arising from a cross-disciplinary community-driven effort (Jones et al., 2017). Both schemas are based on the Schema.org classes SoftwareApplication and SoftwareSourceCode, linking the data to facilitate semantic web discovery. To facilitate this, the FAIR-IMPACT³⁰ project published the Research Software MetaData (RSMD) guidelines focusing on an effective collection and curation of metadata (Gruenpeter et al., 2023).³¹ These guidelines help archiving, referencing, describing and citing software and making its metadata FAIR, benefiting researchers, developers, institutions, publishers, and infrastructure managers by offering tailored best practises.

4.3 Machine-actionable SMPs (maSMPs)

Adopting standardised metadata for SMP has an additional advantage—it allows the automation of initiating processes and tasks in the software development workflows. The RDA DMP Common Standards working group³² defined a machine-actionable (maDMP) to overcome limitations of text-based documents (Miksa et al., 2020). These efforts are now being transferred to SMPs as well. Such an SMP is called a maSMP. Building on the structured metadata (Giraldo et al., 2023), a team from ELIXIR presents a machine-actionable version of their ELIXIR SMP (Alves et al., 2021). This development reuses and harmonises elements from the maDMP, Schema.org, Bioschemas and CodeMeta specifications, while also adding new types and properties (see Figure 2).

Although most of the elements are focused on life sciences, the recommendations are domain-independent. To achieve an alignment between the different parties involved in the existing SMP models and to identify gaps within ELIXIR SMPs, RDMO SMPs and RDA maDMPs, a workshop in the summer of 2023 on maSMPs was conducted in Cologne (Giraldo et al., 2023). As a result, a full metadata model was published, including entities involved in software management planning; such as an SMP itself, software source code, software release, documentation, authors and their relations (Giraldo et al., 2023).³³ This was followed by an NFDI4DataScience hackathon at ZB Med in December 2023.³⁴

5.1 Advantages and disadvantages of SMPs

There are a lot of advantages using SMPs. First of all, information about research software is transferred from tacit to explicit knowledge. Such a superimposition of knowledge offers many advantages. It also becomes clear which aspects of working with research software have not yet been taken into account or have only been discussed incompletely. The direct specification of questions activates such ‘hidden’ knowledge. Writing down the findings then leads to an explicit form of knowledge that can be used in the software project for all participants (Grossmann and Franke, 2023, p. 459–461). Furthermore, more detailed description makes the emerging software perceived as an entity in its own right. It has a name, a management plan with responsibilities, and the like. The software thus becomes an object that can be named directly; unlike implicit parts. At the same time, software thereby also receives more institutional attention and, above all, appreciation. An SMP can significantly accelerate this development. In addition, an SMP can be used in a similar way to a DMP. For example, it is quite possible to use an SMP to convince decision-makers of the excellence of one’s own research idea in the case of a third-party-funded project. In the same way, an SMP could also become a kind of ‘deliverable’ for larger software-using research projects in the future.

There are also disadvantages in using SMPs. First, in general, formalising knowledge is a decision that should be made consciously. An SMP creates explicit knowledge. Especially in the field of software and development, this can be seen as a disadvantage. Agile methods benefit from the ability to iterate and having fewer start-up management methods. An SMP can be a disadvantage here due to the loss of flexibility. Although an SMP costs time at the beginning, the effort actually pays off in the later phases. However, the scientist must already be willing to invest this resource of time unit. And finally, on a fundamental level, it should also be discussed and debated on a fundamental level whether an SMP is necessary at all. Not every small script needs its own SMP. Categorisations of software, such as those proposed by the German Aerospace Center in 2018, can help in deciding how far the effort and return of an SMP are in a reasonable ratio (Schlauch et al., 2018, pp. 7–8).

But, there are also quite different directions in which one could think. Instead of DMP and SMP being separate entities, one could also think about an Reproducibility Management Plan (RMP) or a Data and Digital Object Management Plan (D(DO)MP) for everything (Seibold, 2023; Specht et al., 2023). Such a plan could include everything—text, data and code. Such a plan could include everything—text, data and code. This can be an advantage, as the whole management information concerning all outputs produced within a project remains collected in one document. However, research is often complex. As the size of a scientific project increases, so does the complexity, so that having a central place to manage all the results also leads to a large, voluminous document. In reality, there is then simply a danger that it will hardly ever be read, let alone applied.