Sensor Network Infrastructure for a Home Care Monitoring System

1. Introduction

Ambient assisted living (AAL) solutions are currently becoming more and more common and are quickly moving from research prototype to actual commercial application. A key factor for the success of such systems is the smooth, easy and reliable integration of all hardware components and the integration of the hardware components with the higher level of the system performing monitoring and reasoning on the data. This is especially of importance in home care monitoring, as the system needs to be deployed in a private home and, therefore, requires a quick installation, adaptability to changes in sensor configuration and reliability over long periods of time. The GiraffPlus project is an EU FP7-funded project in which we develop and thoroughly evaluate a complete system that collects daily behavioral and physiological data from distributed sensors, performs context recognition, a long-term trend analysis and presents the information via a personalized interface. GiraffPlus supports social interaction between primary users (elderly) and secondary users (formal and informal caregivers), thereby offering an immediate and obvious benefit, which makes the system attractive and worth using. The GiraffPlus system is named after one of its components: the Giraff telepresence robot. The robot uses a Skype-like interface, allowing caregivers to virtually visit an elderly person in the home. The GiraffPlus system also includes a network of sensors placed in the home. These include physiological sensors for, e.g., weight, blood pressure and pulse oximetry, and environmental sensors. Data from these sensors are stored in a database and processed by an advanced context recognition system, which uses constraint-based temporal reasoning in order to detect events on-line or perform inferences about long-term behaviors and trends. Secondary users can access the data and events remotely through personalized services: users may have access to different data, may be interested in different events and may want the information presented in different ways.

This paper focuses on the architecture of the GiraffPlus system and, in particular, the hardware components and their integration via a middleware infrastructure. It first presents related work, then an overview of the system, the hardware and software components (and, in particular, the middleware), the data storage, the context recognition and configuration planning. Finally, the paper reports on the testing of the system and presents the conclusions.

The key contribution of the paper is the presentation of an implemented system for ambient assisted living (AAL) tested in a real environment. It combines the acquisition of sensor data via a flexible and adaptable middleware with high-level reasoning and, in particular, context recognition.

2. Related Work

The increase in lifestyle related diseases, together with an aging population, are important driving forces to develop systems that facilitate the monitoring of health status independent of location: at home, at work or in the hospital. One important trend is that both the acquisition and processing of physiological data are moving out from the hospitals, allowing patients to stay at home or at work. Furthermore, there is a trend towards technologies that achieve prevention of disease. These technologies have now reached the point where they have become enabling technologies in clinical applications [1]. Substituting traditional sensors by smart textiles for health monitoring has been another hot topic in the last few years [2]. Several research groups and industry have initiated projects in this area, for instance MyHeart ( http://www.hitech-projects.com/euprojects/myheart/), an EU FP6 project with Philips Research as one of the major contributors, focusing on vital sign monitoring, and HeartCycle ( http://heartcycle.med.auth.gr/), also coordinated by Philips, produced a key result in the provision of closed-loop disease management. In the industry, the Continua Health Alliance [3], which started in 2006 to enable an interoperable personal telehealth ecosystem, has become a major force in the personal telehealth domain, and over a dozen interoperable products have already been certified. Continua has also made great progress in defining interoperability for the LANand WANinterfaces and now enables end-to-end interoperability. For the transport level, Continua has recently adopted the ZigBee Health Care Profile ( http://www.zigbee.org/) and the Continua Bluetooth Profile in addition to Bluetooth Low Energy (BLE) and USBand the ISO/IEEE11073 Personal Health Device family of standards [4] for the data level.

In the GiraffPlus project, we aim at deploying a number of sensor devices that are pervasively integrated in the home or that can be used by the elderly to collect vital signs measurements. The difference with the end-to-end architecture of Continua is that we are interested in a holistic view of the home environment, and we aim to collect information potentially coming from devices belonging to different application domains: personal health devices, home automation devices, entertainment devices, smart energy management devices, and so on. The novelty of our approach is in the integration of existing sensors into one system aimed at providing increased home safety and in supporting monitoring physiological parameters over a long period of time. The acquisition of physiological data will include the reliability of non-invasive data acquisition methods and the reliable detection and filtering of measurement artifacts. In wireless sensor networks (WSNs), data aggregation, fusion and filtering are an important topic and combined with aspects, such as security, robustness, reliability and extended lifetime, are very challenging. The level of complexity increases even more considering that sensed data may be originated by subsystems having different constraints and using different protocols. For example, user location can be derived by using hybrid systems [5] based on infrared vision, a radio tag worn by the user and by a number of sensor nodes placed in the environment.

To address these issues, a component-based framework has been developed in the GiraffPlus project. In this framework, different components can be deployed to accomplish its relative task. The aggregation, fusion and filtering of data coming from the WSNs are done in specific components called gateways, which, being aware of the particular technology below them, collect and aggregate sensed data and present it in a common format for all the applications in a transparent way. The context recognition component, for example, can use these data without worrying about the particular communication protocol needed to gather them. In the same way, a specific component offers a secure and reliable channel to gather and store data, hiding all the complexity of handshaking and the replica mechanism to the application components in the common APIexposed by the low level layers (connectors) to the upper layers (applications).

In this context, the integration of all components is done by means of a distributed middleware, which is able to hide the heterogeneity of the sensor devices. This kind of architecture, composed of different layers that communicate through a middleware layer, is becoming increasingly important [6]. Since 1997, the OSGiplatform has attracted much attention among the middleware technologies, thanks to the modularity layer it adds on top of the Java programming language and the service-oriented architecture (SOA) adopted to dynamically bind service components. It has been recognized as the Java middleware, and it became mainstream in 2003, when big players, like IBM and Oracle, started to adopt OSGi as the foundation run-time for enterprise applications (i.e., Eclipse). In the academic world, the SOCAMarchitecture [7] was one of the first to suggest an OSGi-based infrastructure for context-aware applications. In fact, OSGi is suitable for acting as a smart gateway able to link different network technologies, but still providing the easy extendability and configurability of the system, which are valuable properties to meet the interoperability requirements of dynamic environments. For these reasons, OSGi has been chosen as a reference framework in the development of the middleware solution integrated in the GiraffPlus project.

3. Overview of the GiraffPlus System

This section describes the overall architecture of the GiraffPlus system. Figure 1 shows the general component diagram of the GiraffPlus system. In particular, three main components have been identified: (a) Physical environment and software infrastructure; (b) Data visualization, personalization and interaction service; (c) Middleware infrastructure.

The physical environment and software infrastructure component represents the basic level of the functionalities of the GiraffPlus system. All the data services are grounded on functionalities of this part of the system. This module is also in charge of providing the common and interoperable communication service. In particular, the:

• The software infrastructure component includes information management software, storage management software, IT operations management and security software and other infrastructure software. This component provides data integration for achieving the consistent access and delivery of data across the spectrum of data subject areas and data structure types in the system to meet the data consumption requirements of all the application and business processes of the GiraffPlus services.

• The physical environment component includes the sensor network and the Giraff robot. The role of the sensor network is to gather the data generated by the sensors (deployed in the elderly person's home), as well as to provide the rest of the system with the (possibly pre-processed) collected data. The Giraff robot component implements the GiraffPlus tele-presence functionalities, also providing a pilot service to remotely control the robot.

The data visualization, personalization and interaction Service component is the part of the system responsible for creating user-oriented service. A broad way to summarize the module is to provide different end-users with suitable interaction modalities for the available services [8]. The module is subdivided into a front-end part, the interaction service and a back-end part based on two basic services, the personalization and the proactive services:

• The interaction service is the basic front-end of GiraffPlus to all the human users' contacts. It will provide visualization services adapted to the different interactions created within the system. Depending on the classes of end-users, personalized services and specific dialogue boxes are offered, which take into account his/her specific needs/roles.

• The personalization service acts as the back-end of the interaction service and is in charge of collecting and keeping up-to-date all the data needed to generate personalized interactions. It basically creates and dynamically maintains profiles for all the end users involved in GiraffPlus and also provides some reasoning services specifically tailored for the persons involved (e.g., the reminder settings and associated dialogues).

• The proactive services are responsible for collecting specific functionalities to prepare content to be sent from the technological modules to users. Examples of GiraffPlus proactive services can be the reminder and the warning report builder.

Finally, the interconnections among components are also relevant. In this regard, a crucial role is played by the middleware infrastructure component, as it provides the central connection point that is shared by all the components according to the needed information exchanges. It provides to the data visualization, personalization and interaction service (DVPIS) and to any other advanced service to be integrated in the GiraffPlus system a common way to interact with the key functionalities of the system. In particular:

• The service discovery and communication service, providing the primary functions to establish and maintain connections between nodes of the GiraffPlus ecosystem. It enables different services on different nodes to share, modify and manipulate data.

• The storage service, acting as a proxy for accessing the storage management functionalities deployed remotely. Thanks to this service, a generic service built upon the middleware can access historical data and configurations stored on the database without caring about connectivity and how to access it.

• The intelligent monitoring and adaptation service act as a proxy for accessing the context/activity recognition and configuration functionalities deployed on a dedicated server. As for the storage service, this component is necessary to avoid that other high level components must be aware of connectivity and access settings.

The entire data and workflow in the GiraffPlus system uses the middleware infrastructure to realize a distributed assistive system. For instance, the sensor network provides the data storage with the data collected through the sensors and that need to be placed in the database; the context inference system retrieves the sensors data from the data storage and exploits them in order to reason about the person/ambient status; the configuration planning system selects the set of sensors needed to implement a requested monitoring activity. The personalization service dynamically provides user profiles that are stored in the database. All these actions are done interacting with the middleware infrastructure that enables independently designed applications, software components or services to work together, supporting data consistency, composite application and multi-step process styles of integration by connecting various, heterogeneous devices and services to a single, unified network.

Part of the system is implemented using a cloud computing solution. There is a centralized database, and some services (intelligent monitoring and adaptation service) are running on the GiraffPlus cloud infrastructure and not locally in elderly person's home. These modules are represented in Figure 1 as part of the software infrastructure component.

4. Hardware Components

In the following section, we briefly describe the most important components used in the system. We first consider the sensors that are used for monitoring physiological parameters, then the sensors for monitoring activities and behaviors and, finally, the sensors used to alert emergency situations. The sensors have been selected according to the results of a user requirement study involving both primary and secondary users that have identified the activities and data that are important to monitor. The Giraff robot is also presented.

5. The Middleware

GiraffPlus is a set of software modules that helps to build so-called “assistive systems” by connecting various, heterogeneous technical devices into a single, unified network. In GiraffPlus, a single device that is connected to the system is referred to as a “node”. There are two ways to integrate a device into the GiraffPlus system, assuming that the device in question is networked (can send and receive data using a network protocol, either wired or wireless). The first way is to install a specific piece of the GiraffPlus platform on the device, the so-called “middleware” (as in the case of the Giraff robot). The middleware software contains the communication infrastructure of the GiraffPlus platform, and all devices that run the middleware can actively participate in the communication of the system. The second way of connecting devices to the GiraffPlus system does not require a given device to run the GiraffPlus middleware. The device in question is rather connected to a node that runs the middleware, and this node is used as an intermediary by the system in order to control the additional device. For many devices, such as low-power wireless sensors, this is the only possible way of connecting them to the system, simply because they cannot run any additional software beyond their firmware. Although these slave-devices cannot actively participate in the communication with the rest of the system (as they are just queried for data), their advantage over regular nodes is that they can simply be “plug-and-played” into a running system.

The GiraffPlus middleware is inspired by the concrete architecture of the UniversAAL [9] middleware and partially derived from the PERSONA [10] project. The UniversAAL ( http://universaal.org/) project is the most recent European initiative aimed at delivering a software platform for AAL, which started to consolidate the architecture and software developed in recent research projects, like PERSONA http://www.aal-persona.org/, SOPRANO http://www.soprano-ip.org/), OASIS( http://oasis-project.eu/), and mPOWER( http://www.sintef.no/Projectweb/MPOWER/). One of the major tenets of the UniversAAL approach is to separate the reference architecture used to lay out the smart-home environment from the concrete architecture and implementation used to realize the intelligent services. This abstraction allows the design of different platforms, for example the ones based on different paradigms and technology, but still sharing common building blocks. In this way, common building blocks enable the creation of an ecosystem by ensuring the interoperability at the level of the interface or functional services. One of the main goals of the GiraffPlus middleware is to be compatible at the level of reference architecture with UniversAAL-based systems. This means that with some simple adapter, the components used in GiraffPlus will be able to run in a UniversAAL-based system.

Besides the middleware, the GiraffPlus platform is made up of various other software parts, which we will refer to as “higher level components”. The middleware is the basis for all these higher level components. Components rely upon it, as the middleware is capable of hiding the distribution and heterogeneity of the diverse devices that make up the system. From an application programmer's perspective, this means that he does not need to worry about the fact that some of the applications (and higher level platform components) involved in the workflow may actually be running on other devices. The application simply forwards all of its messages and requests to the middleware component, which makes sure that each message reaches its recipient. The middleware does, of course, not hinder any application from accessing a device's operating system API. It simply eliminates the need for resolving dependencies.

Hiding the distribution and heterogeneity of the devices is actually just a part of the middleware's job. The middleware infrastructure is composed by different modules and connectors offering APIs to access the main functionalities of the GiraffPlus system. Its architecture presents two main layers implemented as a set of OSGi bundles: a connector layer and a module layer. The architecture of the middleware infrastructure is reported in Figure 3.

The connector layer provides a pluggable mechanism in order to enable communication capabilities among nodes and the possibility to access remote resources via RESTful (representational state transfer) interfaces. The connector layer is a set of artifacts that make use of third party libraries. Some notable examples are MQTTfor the communication connector and Jersey for the RESTful connector.

The module layer is designed in order to detach the connectors from the rest of the architecture by providing a connector-independent set of APIs. The GiraffPlus middleware implements a specific module for the service discovery and communication, intelligent monitoring and adaptation and storage capabilities:

• The service discovery and communication module is responsible for ensuring that communication can take place across all the networked nodes. To do this, nodes must firstly be discovered and enabled to exchange messages. The communication building block defines how data is transported from the sender to the receivers. For the data exchange between the communicating parties, different communication modes (publish/subscribe and REST methods invocation) may be supported. Additionally, possible roles that are involved in the communication (e.g., publisher and subscriber) are specified.

• The storage module is responsible for providing a general database service for all the data generated by parts of the system and providing data access functionalities. Specifically, the role of this component is to manage a database containing all the data collected through the middleware service and generated by other system's components (for instance, the sensor network). Additionally, it enables other components to access the information and reason about it (also considering historical evolution). This module guarantees permanent data to be gathered, as well as offering the general possibility to use the future long-term experimentation as a benchmark gathering.

• The intelligent monitoring and adaptation module is the component responsible for context/activity recognition and configuration planning. This part of the system encompasses two general reasoning systems, namely:

  –

  The context inference system, which is in charge of implementing the requested monitoring activities by means of context/activity recognition and relies on a timeline-based representation of the data generated by the sensors.

  –

  The configuration planning system, which is responsible for providing suitable configuration settings for the sensor network according to the requested monitoring activities.

The details about the middleware interfaces and the functionalities of each module are illustrated in the following sections.

6. Service Discovery and Communication

Within the proposed AAL ecosystem, an AAL space is intended to be the physical environment (such as the home of an assisted person) in which independent living services are provided to people that need any sort of assistance. In such a virtual ecosystem, hardware, as well as software components can “live” while being able to share their capabilities. In this space, the proposed platform facilitates the sharing of three types of capabilities: service (description and discovery of components), context (the database on shared models) and control (the state of the component of the system). Therefore, connecting components to the platform is equivalent to using the brokerage mechanism of the middleware in these areas for interacting with each other. Such connectors together with the application logic behind the connected component are called altogether AAL services [11], as at its core, an assistive system is nothing else, but a local private network of technical devices. When additional nodes try to (re)join the system, the discovery mechanisms of the middleware automatically recognize and integrate all qualified nodes within reach. The GiraffPlus middleware has different mediators to discover nodes and reach them, and each of them is responsible for the delivery of a certain message category: the buses. On these buses, a listen-announce protocol has been implemented to let nodes be discovered. A publish-subscribe pattern is used to send and receive messages.

7. The Mobile Middleware

In order to collect and aggregate data from physiological sensors (Section 4.1) and to give the primary user a personal device connected to the GiraffPlus network, a mobile version of the middleware running on Android platforms has been developed. The mobile middleware has been designed aiming at optimizing both the performances and the modularity to reflect the OSGi-based reference architecture. It is made of two components running as services in the background and providing the required functionalities on demand. These two components, shown in Figure 6a, are the middleware and the CommunicatorConnector. On top of them, several services (Android applications) can be created to send or receive sensor data through the network by interacting with other devices.

The scope of the middleware component is to provide to the upper level applications a common interface, compliant with the OSGi version of the middleware, for discovering other sensors/services in the network, subscribing in order to receive their updates as consumers or publishing data as producers. The CommunicatorConnectorcomponent is used to decouple the business logic contained at the middleware level (handling the control and context information) from the actual mechanism used to exchange messages in the network. This module is in charge of dispatching and delivering the messages based on a given communication protocol.

To guarantee the same level of abstraction between the middleware modules, both the middleware and the CommunicationConnectorcomponents are implemented as bounded services, a particular type of service that exists only when serving another application. The inter-process communications among the components are performed using a well-specified interface defined with the Android Interface Definition Language (AIDL). The AIDL interface definition has been used to handle multithreading, which means potentially concurrent IPCson the same service.

In the proposed system, the IMiddleware interface (Figure 6b) offers the same set of functionalities to the upper level applications of the OSGi version, allowing a GiraffPlus component to be easily ported to the mobile platform.

8. Data Storage

The guiding principles in the design of the GiraffPlus Long-term Data Storage (LTS) system have been security, flexibility, reliability, efficiency and scalability. Security is paramount, as we are dealing with sensitive personal health information. Flexibility is important, as GiraffPlus is a research project, and it is impossible to foresee all the possible data structures that might, at some point, be stored in the system, as well as all possible connections between them. At present, two types of data are stored in the database: the home configuration data, which is needed for middleware auto-configuration, and sensor data from all sensors in all GiraffPlus-enabled homes. The storage system needs to be able to reliably store vast amounts of sensor data and enable users and other GiraffPlus components to efficiently access the stored data. Last, but not least, the system needs to be scalable, as we expect the amount of data to vastly increase during the course of the project and after, during the commercialization phase of the project.

Apart from the GiraffPlus middleware storage module, which enables secure access to stored data via the middleware, the GiraffPlus Long-term Data Storage system is composed of four components:

• the database, which stores the actual data,

• the GiraffPlus MQTT listener component, which is connected to the central MQTT broker and forwards all relevant data to the database in a secure fashion,

• the GiraffPlus LTS RESTful web service, which enables secure access to the database,

• the GiraffPlus engineer GUI, which enables GiraffPlus professionals to enter configuration data for individual homes,

• the GiraffPlus Certificate Agency, which issues certificates used to authenticate various users and components in the GiraffPlus ecosystem.

9. Context Recognition and Configuration Planning

The responsibility of the context recognition service (CRS) within the GiraffPlus system is to produce timelines containing inferred activities from the collected sensor data and relay these to the visualization systems, which, in turn, can present them to authorized users. These timelines are produced “off-line” on demand, i.e., as a response to a query from a user who wants to view the activities of the primary user over time. Characteristic of this approach is the fact that it can take into account future data and use that to revise past inferences, which can be altered or extended as new data arrives, while an “on-line” approach, as the name suggests, continuously commits to and inferred activity for the current point in time which will not be revised in the future, even if presented with contradictory data. Another responsibility of the CRS is to raise alarms when specific conditions are met (e.g., the primary user has been staying in the bathroom for an abnormal amount of time, which might indicate a fall). Alarms are implemented by continuously querying the context recognition system at regular intervals (in this case, the system only takes into account past data to infer the presence of an alarm at a given point in time).

10. Testing of the System in Real Homes

The system is currently being tested in six real homes (two in Sweden, two in Spain and two in Italy), where the system has been deployed and will be used for one year. The system will be placed in nine additional homes in the spring of 2014. A map of one of the homes in Sweden with the positions of the sensors installed is shown in Figure 9. This particular apartment is inhabited by an 82-year-old man (born 1931) who has been living alone since his wife departed two years ago. He had a stroke two years ago and spends most of his time inside; the exceptions are when he goes outside to do shopping or to visit any of his three sons with his mobility scooter.

The sensors that are currently present in the apartment are the ones previously described. The Giraff robot present in the apartment is an updated prototype version of the robot equipped with a touch screen. As we previously stated, a key factor for the GiraffPlus system is the smooth, easy and reliable integration of all hardware components and the integration of the hardware components with the higher level of the system performing monitoring and reasoning on data. The test site was used to validate this aspect. Once the sensors are physically installed in the home, an operator simply configures them on the GiraffPlus Engineer GUI (Section 8.4), specifying the locations; then, the system automatically recognizes and exposes them through the middleware on the right topics. Henceforth, any application deployed in the home can gather and use sensed data.

11. Conclusions

In this paper, we presented GiraffPlus, an implemented system for supporting independent living for the elderly. The system includes many components, from sensors to high-level reasoning. A middleware is used to integrate them and to make the addition and removal of components easy. We identified the crucial features of the system in the easy and reliable integration of all the hardware and software components, the possibility of the quick installation of new hardware and software modules, the adaptability to make changes in the configuration and the reliability over long periods of time. These features have been tested in a real home environment. This is another key aspect of the proposed solution. In fact, the system has been tested and is currently deployed in real homes across Europe for the long-term monitoring of physiological data, the activities of daily living and the behaviors of elderly people.