1 Introduction

Energy efficiency and low-carbon technologies are key enablers to mitigate the increasing emission of green-house gases still resulting in a global warming trend [1]. The efforts to reduce green-house gas emissions also strongly affect the power system. Renewable sources, storage systems and flexible loads provide enhanced possibilities but power system operators and utilities have to cope with their fluctuating nature, limited storage capabilities and the typically higher complexity of the whole infrastructure with a growing amount of heterogeneous components [2]. Additionally, due to changing framework conditions, like the liberalization of the energy markets and new regulatory rules, as well as technology developments (e.g., new components), approaches for design, planning, and operation of the future electric energy system have to be restructured. Sophisticated component design methods, intelligent information and communication architectures, automation and control concepts as well as proper standards are necessary in order to manage the higher complexity of such intelligent power systems (i.e., smart grids) [3–5]. Besides technical challenges also economic, ecological and social issues have to be addressed in smart grid research and innovation, too.

During the last decade—especially in the past framework programs of the European Commission (i.e., FP6 and FP7)—a growing number of research and technology development activities have already been carried out in this area. Their main attempt was to fulfil the challenging goals and needs of the Strategic Energy Technology Plan (SET-Plan) of the European Commission for a sustainable environment and to foster the innovation in cost-effective low carbon technologies [2]. Several research and demonstration projects (Grid4EU, EvolvDSO, IGREENGrid, etc.) have already shown the applicability of the proposed approaches.

Due to the considerable higher complexity of cyber-physical energy systems (i.e., power system + automation/Information and Communication Technology (ICT) + services) [6] the validation of smart grid configurations will play a major role in future technology developments. Today, the first demonstration projects for smart grid technologies and solutions are successfully finished. The probability that the key findings and achievements will be integrated in new and/or modified products, solutions and services of the European manufacturers is now very high. Therefore, the expected large scale installation and roll out of smart grid technologies will require the corresponding validation methods and tools; this is identified as a major European infrastructure priority for the near future [7, 8].

There is a lack of an integrated approach for analysing and evaluating smart grid configurations addressing power system, as well as information, communication and automation/control aspects. Validation approaches available today, which are focused mainly on device/component-oriented methods [9], are mandatory but not enough. In order to guarantee a sustainable and secure supply of electricity in a more complex smart grid system as well as to support the expected forthcoming large-scale roll out of new technologies, proper validation and testing methods are necessary addressing the power distribution system in a cyber-physical manner.

The aim of this paper is to analyse different validation approaches taking virtual-based (simulation, software-in-the-loop, co-simulation), real-world based (laboratory, field), and a combination of both (controller-hardware-in-the-loop, power-hardware-in-the-loop) approaches into account. In addition, necessary future research directions are discussed.

The rest of this paper is organized as follows: Sect. 1 deals with the increasing need for system-level validation approaches applied across the development process of smart grid systems. A brief overview of existing validation and testing methods is provided in Section 3. In Sect. 1 necessary future research directions are derived and discussed. Finally, the paper is concluded in Sect. 5 with the key findings and a brief overview of planned future work.

2 Needs for system-level validation

Compared to traditional planning and operation of power systems, future smart grids tend to have a higher complexity mainly due to their cyber-physical nature. In this section the necessary intelligence on different levels as well as different phases of the design and development process of smart grid systems are discussed. Moreover, needs and requirements for validating such approaches are raised.

2.1 Intelligence on different levels—higher complexity of future power systems

Power system operation is of vital importance, and has to be developed far beyond state-of-the-art to meet future challenges (high penetration of renewables, distributed storage systems, e-mobility, etc.). In fact, nearly all European countries faced an abrupt and very important growth of Renewable Energy Sources (RES) such as wind and photovoltaic that are intrinsically variable and up to some extent difficult to predict. In addition an increase of new types of electric loads such as air conditioning, heat pumps, and electric vehicles; and a reduction of traditional generation power plants can be observed.

These facts have increased the level of complexity of system operation and management. To avoid dramatic consequences, there is an urgent need for system flexibility increase and full implementation of smart grids solutions such as information and communication technology and power electronic-based grid components in order to allow a number of system functionalities, such as power/energy management, demand side management, ancillary services, etc. This also requires distributed intelligence on different levels in the system as shown in Fig. 1.

Fig. 1.
figure 1

Intelligence on different levels applied to smart grid systems (adopted from [10])

Full size image

A brief categorization of the different levels of intelligence applied to smart grids systems can be carried out as follows:

  • System level: Operational approaches like power utility automation, demand-side management or energy management are tackled by this level. Functions and services of the underlying sub-systems and components are triggered in a coordinated manner for execution from a systems perspective. Both, central as well as distributed control approaches are used on this level.

  • Sub-system level: The optimization and the control of sub-systems are carried out below the system level whereas the corresponding functions, services, and algorithms have to deal with a limited amount of components (DER, energy storage system, electric vehicle supply equipment, etc.). Examples for this level are micro-grid control approaches and home/building energy management concepts. Also an energy storage system together with a distributed generator installed at the customer side can be considered as a sub-system. Distributed automation and control are typically applied.

  • Component level: Distributed Energy Resources (DER)/RES, distributed energy storage systems but also electric vehicle supply equipment is covered by this layer. Components typically provide advanced functions like ancillary services. Intelligence on this level is either used for local optimization purposes (device/component behaviour) or for the optimization of systems/sub-systems on higher levels in a coordinated manner.

  • Sub-component level: Intelligence on this level is mainly used to improve the local component behaviour/properties (harmonics, flicker, etc.). Power electronics (and their advanced control algorithms) is the main driver for local intelligence on this level. The controllers of DER, distributed energy storage systems, electric vehicle supply equipment and other power system components (tap-changing transformers, FACTS, etc.) can be considered as examples for sub-components.

Major requirements for the realization of ICT/automation systems and component controllers are flexibility, adaptability, scalability, and autonomy. Furthermore, interoperability and open interfaces are also necessary to enable the above described functions on the different levels. [10]

As a result of these developments and the availability of advanced services and functions, smart grid systems tend to have a higher complexity compared to the traditional power system configuration.

2.2 Design and development process of smart grid solutions

Usually, the design and development process of smart grid solutions covers several stages. They are mainly dependent on the applied system engineering approach or process model (V-model, etc.), but also on the overall complexity of the system under development. In general, the following four main design stages can be observed during the whole development process [11]:

  • System-level requirements and basic design: During the first design stage usually the system-level requirements and application scenarios are being identified. In the following a basic design and high-level architecture specification are typically been carried out.

  • Detailed design: After the conceptual design has been elaborated a detailed design and engineering of the system under development is done. Functions and services are also identified and specified.

  • Implementation and prototype development: During this development phase first prototypes are being developed. The process of transforming a concept into a prototype often introduces issues which were not considered during the design stage(s). Often boundary problems like communication latencies or nonlinearities are neglected during the first versions of a basic concept. During the development of a prototype iterative refinements of solutions/algorithm are often necessary.

  • Deployment and roll out: This stage mainly covers the realization of a product as well as the installation/roll out of components and solutions in the field.

Compared to other domains, challenges during the design and development of smart grid solutions are (i) the fulfilment of high-reliability requirements, (ii) the observance of (strict) real-time requirements, (iii) the compliance with national rules, and (iv) the interaction with several system integrators/manufacturers. Therefore, proper design and validation methods are required in order to prove the outcomes and results of the different stages. Due to the higher complexity of smart grid systems advanced testing methods are necessary addressing cyber-physical issues. The needs and requirements for such new approaches are discussed below.

2.3 Needs and requirements for a holistic system validation procedure

Validating and testing smart grid technologies and developments are tasks which require a holistic view on the overall development process. The entire domain spectrum of future smart grid solutions has to be taken into consideration. Besides technical components such as grid infrastructure, storage systems, (distributed) generators, loads, etc. it also comprises customers, markets, ICT, regulation, governance, and metrology to name a few. The full development process has to be covered, too. This includes design, analysis, testing, verification (even certification), as well as deployment. Even more, the whole range of aspects from interest and relevance for a stable, safe and efficient smart grid system has to be regarded. Thus, small-signal stability together with large-scale scenarios, short-term impacts and long-term sustainability, economic feasibility/profitability, and cyber-security have to be analysed in an integrated manner.

It has to be mentioned that comparable processes have already been successfully implemented in other application domains like automotive, consumer electronics, mechanical/chemical engineering (albeit on an arguably less complex level) [12]. The power and energy system domain can profit from existing approach and can adapt them to fulfil needs and requirements of the domain. There is no need to start from scratch.

In order to realize a sustainable and cost effective holistic procedure in smart grid system validation, the following needs can be identified [13]:

  • A cyber-physical (multi-domain) approach for analysing and validating smart grid solutions on the system level is missing today; existing methods are mainly focusing on the component level—system integration topics including analysis and evaluation are not addressed in a holistic manner.

  • A holistic validation framework (incl. analysis and evaluation criteria) and the corresponding research infrastructure with proper methods and tools need to be developed.

  • Harmonized and possibly standardized validation procedures need to be developed.

  • Well-educated professionals, engineers and researchers understanding smart grid solutions in a cyber-physical manner as well as their validation requirements need to be trained on a broad scale.

2.4 Illustrative example

In order to get a better understand about future system validation needs an illustrative example addressing coordinated voltage control in a power distribution grid is introduced. According to Fig. 1 an On-Load Tap Changing (OLTC) transformer is used in this smart grid solution together with reactive and active power control provided by DERs and electric storages. The goal of this application is to keep the voltage in the power distribution grid in defined boundaries due to distributed generation and to increase its hosting capacity with a high share of renewables [14]. The corresponding control approach has to calculate the optimal position of the OLTC and to derive set-points for reactive and active power which is communicated over a communication network to the DER devices and electric storages.

Before installing this smart grid solution in the field various tests need to be carried out. This includes the validation of the different components (incl. local control approaches and communication interfaces) on the sub-component and component level. The OLTC control approach needs to be tested, too.

Nevertheless, the integration of all components and sub-systems is one of the most important issues. The proper functionality of all components is not a guarantee that the whole system is behaving as expected. As expressed above, a system-level validation and testing is necessary in order to prove that the whole power system application together with the ICT devices is working properly and as expected.

3 Overview of existing validation approaches

Having identified open issues in validation and testing of future smart grid systems this section provides an overview of available approaches. The usage of pure virtual-based methods, real-world-based methods, and also a combination of both is briefly discussed. Moreover, a mapping of these approaches across the four different development phases from the last section is provided.

3.1 Software simulation

Simulation-based approaches are very common in power systems engineering. Individual technological areas (power system, ICT/automation) have been analysed in dedicated simulation tools. For example, transient stability and steady state simulations are very often used to investigate the behaviour of power systems and their components. Different approaches and corresponding tools have been developed over the years and are nowadays state-of-the-art [15]. Similar developments can be observed also in the domain of ICT and automation systems.

However, the development of smart grids solutions and technologies urge for a more integrated simulation approach covering all targeted areas [16]. Nowadays, the usage of simulation as development approach gets more of interest [17, 18]. Analysing the behaviour of smart grid systems requires hybrid models combining continuous time-based (physics-related) and discrete event-based (communication and controls-related) aspects.

Co-simulation (or co-operative simulation) is an approach for the joint simulation of models developed with different tools (tool coupling) where each tool treats one part of a modular coupled problem. Intermediate results (variables, status information) are exchanged between these tools during simulation where data exchange is restricted to discrete communication points. Between these communications points the subsystems are solved independently [19]. Thus, the term co-simulation refers to joint simulation of several aspects or domains of a system. Co-simulation takes under consideration the complexity of the simulated system and influences between different aspects or domains interconnected in the same system. The term co-simulation was used to refer to hybrid hardware/software simulation systems, as defined in [20], but it can also refer to a set of interconnected software simulations. It can be distinguished between hardware/software co-simulation, in specific cases referred also to as hardware-in-the-loop [21], and purely software co-simulations. Recent work on power system co-simulation includes efforts to integrate communication, automation and control but also electricity market topics [22].

3.2 Lab-based testing

Laboratory experiments in electrical engineering for testing or certifying single or small setups of components are nothing new. However, the decentralization of operation and control as well as the massive deployment of ICT components (and thus introducing shorter innovation and product cycles than hitherto known in energy supply systems) drastically increase the complexity of the system under investigation and easily exceed the scope of existing laboratory setups. In a single laboratory environment the evaluation of holistic cyber-physical energy systems is out of the question leaving simulation (or hybrid co-simulation-experimentation incl. hardware-in-the-loop) setups as the only viable option. Moreover, flexibility to deploy intelligent algorithms in different locations across the system is also necessary to move towards a lab-based testing of integrated power systems.

3.3 Hardware-in-the-Loop (HIL) experiments

Nowadays HIL-based approaches get more interest from the power system domain. Two different approaches can be distinguished, namely Power-Hardware-in-the-Loop (PHIL) and Controller-Hardware-in-the-Loop (CHIL).

PHIL testing is an advanced tool for power system analysis, testing and validation and is considered the upcoming-future methodology for testing smart grid systems and components (DER devices, energy storages, and Distributed Flexible AC Transmission System (D-FACTS), etc.). It performs the connection of an actual power device or system (the Hardware under Test (HuT)) to a real-life system which is simulated in a Digital Real-Time Simulator (DRTS), allowing repeatable and economical testing under realistic, highly flexible and scalable conditions [23]. Extreme conditions can be studied with minimum cost and risk, while problematic issues in the equipment behaviour can be revealed allowing an in depth understanding of the tested device. PHIL testing combines the benefits of numeric simulation and hardware testing and is constantly gaining interest at international level. It is particularly suitable for studying the integration of new components in distribution grids, as physical photovoltaic generators, wind turbines, electric vehicles, energy storages, or whole microgrids. It can be connected to a simulated active network, containing various simulated DER devices, where complex interactions can be identified [21, 23, 24]. However, PHIL is mainly used for testing of single components. A move towards integrated system-level testing (power and ICT sub-systems) using the PHIL approach is necessary.

Besides PHIL also the CHIL approach is of interest for power systems engineers. Instead of having a real power hardware component connected to a simulated power system, only the controller is available as physical device.

From a system perspective, typically offline digital simulations are performed to investigate power system phenomena using mathematical models. However, the large scale deployment of complex devices such as power electronics-based DER with advanced functionalities poses new challenges for simulation, as these devices are particularly difficult to model accurately and might be involved in complex interactions within the power system. Therefore (P)HIL testing can reveal system related phenomena that are not visible in pure digital simulations [24].

3.4 Field trials and large-scale demonstration projects

Besides simulation and lab-based validation approaches (incl. HIL) field trials and large-scale demonstration projects are also of importance for the validation of new architectures and concepts. They have the advantage to test industrial-like prototypes and developments under real-world conditions but a huge amount of preparation and planning work is necessary to realize such kind of field trials. Usually, they are also quite expensive and resource intensive.

3.5 Summary

Table 1 summarizes the above discussed approaches and provides a brief overview about the suggested usage of them across the design and development process (as described in Sect. 2.2).

Table 1. Brief overview of validation approaches used in power system engineering
Full size table

4 Future research directions

As shown in the last section there are already a lot of different possibilities to validate smart grid solutions but there are still several issues which need to be addressed by future research. In the following sections open issues and research directions are outlined.

4.1 Integrated methods and tools

Current developments in the field of smart grids show that future systems will contain a heterogeneous agglomeration of active components often based on power electronics and passive network components coupled via physical processes and dedicated communication connections with automation systems (SCADA, DMS) and advanced metering/measurement systems. Also new applications and services are under development helping to fulfil future requirements and needs.

In order to analyse and evaluate such a multi-domain configuration, a set of corresponding methods, procedures, and corresponding tools are necessary. Usually, pure virtual-based methods are not enough for validating smart grid systems, since the availability of proper and accurate simulation models cannot always be guaranteed (e.g., inverter-based components are sometimes very complex to model or it takes too long to get a proper model). Simulation and lab-based validation approaches have to be combined and used in an integrated manner covering the whole range of opportunities and challenges. Such an approach is necessary when answering system level integration and validation questions.

Figure 2 sketches this idea where a flexible combination of physical components (available in a laboratory environment) and simulation models are combined in a flexible way in dependence of the corresponding validation or testing goal. Such an approach needs the improvement of available methods and tools. In addition, proper interfaces need to be provided as well. There is still space for future research related to this topic.

Fig. 2.
figure 2

Smart grid validation approaches using virtual and real components (adopted from [25])

Full size image

4.2 System level validation procedures and benchmark criteria

A cyber-physical (multi-domain) approach for analysing and validating smart grids on the system level is missing today; existing methods are mainly focusing on the component level—system integration topics including analysis and evaluation are not addressed in a holistic manner. As described above, a holistic validation framework (incl., analysis and evaluation criteria) and the corresponding research infrastructure with proper methods and tools need to be developed.

However, harmonized and possibly standardized validation and testing procedures need to be developed, too. Up to now no standards for power system testing exist. Furthermore, corresponding benchmark criteria are also necessary in order to compare test results and to derive the right conclusions out of performed tests.

4.3 Advanced research infrastructures

Research and laboratory infrastructure for testing power system component developments are widely used. However, infrastructures allowing a flexible and systematic combination of power system simulation with control systems simulation/emulation and full-scale hardware testing of components are not common up to now. For this purpose combined hardware and software testing solutions (co-simulation, CHIL, PHIL) need to be developed and implemented. Also the linking of existing infrastructures as well as the establishment of clusters of them should be in the focus of future research. Such integrated research infrastructure should be able to provide advanced validation and testing services fulfilling future validation needs in a cyber-physical manner.

4.4 Education and training

Supporting the coordinated and structured smart grid technology development and large-scale roll out of solutions which are expected for the near future, a key element in this context is the human factor. Well-educated professionals, engineers, and researchers understanding complex smart grid configurations as well as their validation in a cyber-physical manner need to be trained on a broad scale.

The development of proper training/education programs, summer schools, and corresponding courses could help to educate people working the domain of smart grids. These training possibilities allow power system and Information and Communication Technology (ICT) professionals, (young) researchers and students to improve their understanding of system aspects, technology developments and deployments into the field.

5 Conclusions and future activities

In the upcoming years a large-scale roll out of smart grid solutions, technologies, and products can be expected but up to now no integrated approach for analysing and evaluating smart grid configurations addressing power system, as well as information, communication and automation/control topics is available. The integration of cyber security and privacy issues is also not sufficiently addressed by existing solutions. In order to guarantee a sustainable and secure supply of electricity in a smart grid system with considerable higher complexity as well as to support the expected forthcoming large-scale roll out of new technologies, proper concepts, methods and approaches are necessary to support system analysis, evaluation and testing issues.

Several useful approaches already exist including simulation-based methods, hardware-in-the-loop approaches, and also lab-based testing. However, an integrated approach, the corresponding methods and tools are partly missing up to now. Moreover, research infrastructures need to be further developed integrating a holistic, cyber-physical view onto smart grid configuration; the corresponding training and education activities need to be developed, too.

Some activities have already started on European (FP7 ELECTRA IRP, H2020 ERIGrid) and international level (IEA ISGAN/SIRFN). They are working on a holistic validation procedure and improve existing co-simulation and HIL-based methods but there is a lot of space for future research.