A Collaborative Workflow for Computer-Aided Design in Ambient Assisted Living: The ASIM Project

3.1 Objectives and Partners

This section presents our experience of the ASIM project.

This project addresses the e-Health domain. It is designed to control home automation processes and to assist the monitoring of elderly people. The principle is based on the collaboration of various experts: doctors, occupational therapists, family members, user, installer, etc. They fill multiple parameters regarding the final user, and thus they enable the complete generation of the control/command application for the latter. The monitoring measures can be remotely planned and directly carried out by the elderly person, if possible. This approach seeks to reduce drastically the important costs of e-Health, in order to make it more accessible, and achieves this by reducing specialists’ home interventions.

To achieve this goal, the LabSTICC laboratory and its partners, namely VITY Technology (for home automation control) and Kaptalia (for health sensors), are developing together a multiprotocol server (SMM) embedding health sensors and an easy-to-configure UI. This feature enables the dynamic generation of a usable application dedicated to an individual elderly person, hence providing a solution that takes into account the particularity of each person.

3.2 Scenarios and Requirements

The ASIM project involves two health centers as basic requirements. On the one hand, the Vannes Health Hospital (http://www.ch-bretagne-atlantique.fr/) has provided the scenario for e-Health using a hospital. On the other hand, the Kerpape Rehabilitation Center (http://www.kerpape.mutualite56.fr/) has provided the scenario for people with cognitive and/or motor disabilities who can return home. This approach gives us three main scenarios:

• e-Health with hospital monitoring: The patients return at home after hospitalization; their health condition no longer necessitates critical monitoring, but they require distant monitoring whereby health data are handled by the hospital staff. In this scenario, nurses visit the patients or call them to take health measurements (non-invasive arterial pressure oxygen saturation, electrocardiogram, etc.). A Hospital e-Health dedicated server collects the data that are protected for medical secret. It is the hospital personnel who are responsible for dispatching and interpreting the protected data. A VITY Application server collects the alerts and data concerning the home control.

• e-Health without hospital monitoring: The patients do not require advanced medical monitoring, but an auxiliary nurse or family members are involved. Only a VITY application server is used to handle the classic alerts (falls, wandering, etc.) and any home control alerts. The patients or the family members can use the data for their own concern. Health parameters can be exploited through a green, orange, or red smiley for legal reasons. Generally, it is the family (for an elderly person), or an occupational therapist (after rehabilitation), or a health personnel member (retirement home, generalist doctor, etc.), or even the patients themselves if they are able to, who will set up the ASIM system in the Definition phase.

• AAL: People who require neither hospital nor medical monitoring can use the system for home control, comfort, energy, or security concerns. In this case, the individuals are as much administrators as users.

Note that the handicap or illness is a transverse parameter to these scenarios. Depending on the handicap, the user or a responsible person can set up the system. Depending on the patient’s state of health, the experts are going to collaborate on the selected scenario.

3.3 Multiexpert Collaboration

Currently, in the e-Health domain, most research efforts appear to focus on problems from a technical point of view that is necessary, but not sufficient. For instance, current approaches focus on a comfort maximization system, neural network, fuzzy logic control, advanced user service auto-recognition, low fall detections, etc. Most systems, however, do not take into account the patient’s illness evolution, or the evolution of the user’s needs, which require adapting the system gradually. Changes to the hardware or to the system services imply modifications to the UI, but also to the control/command system.

The ASIM project aims to help directly the patient/user at the source and requires the collaboration of experts to achieve health recovery while promoting the patient’s return home. The health process leading to a home return is complex because it includes multiple stakeholders who are experts in their field.

Thus, the collaborative approach proposes to collect multiple expert points of view through different parameters. Experts include, but are not limited to, doctors, occupational therapists, family members, installer, and final user. This collaboration enables the dynamic generation of a usable application dedicated to an individual person because any illness is particular to each person.

We also include conventional concepts and add the principle of close relatives through a contacts list, the planning of medical intervention from a nurse, and the global warning system that can alert a contact and start an action or scenario after a specific medical event or other home event.

The participative multiexpert collaboration approach is presented in Figure 1. Actors shown on the perimeter of the circle interact on the Generation Server interface to set up the user’s dedicated parameters. They all fill the questionnaire form concerning their domain. Experts interact together or separately depending on the situation and the opportunities. The main advantage is to promote discussion and to raise consensus. The first action starts by selecting a global ASIM scenario, in the outer ring.

Figure 1:

Multiexpert Collaboration to Generate the Final User Application.

In the middle ring, the collaborative approach is used to separate the concerns. The experts enter information concerning the patient: the person and his/her personal links (CNT model), environment (ENV model), interactions, control/commands and scenarios (INT model), medical, energy, or home alerts (CNT model), and the UI aspects (HCI model).

In the inner ring, only an installer can describe a technical solution independently from the multiexpert decisions. Hardware protocol diversity is a recurrent problem in home automation. An installer could integrate various target systems from KNX, ZWave, EnOcean, IP, 868 MHz, 434 MHz, etc., which must be handled specifically. Most technical models are inferred from previous middle ring models (Home, Alerts, Scenarios, Equipments, Interfaces, Language, Cat_Marques, Cat_Equip) after which the installer configures the runtime data of the specific equipment.

Any one of the experts can, after all phases have been completed, generate the final user application and transfer it to the smart system target for use.

3.4 System Architecture

The global system architecture, in Figure 2, describes the steps required to gather the expert views and update the final user application that has to respond to the user’s specific situation.

Figure 2:

Global Architecture with the Application Server (Application Generator) and the ASIM Server (Applications and Home Controller).

From the doctor’s diagnosis, the installer’s specification, the occupational therapist’s specifications, the user’s needs, and the decisions mutually taken, the experts first define (i) their specific parameters through the questionnaire form that is available on the application server on the Web (see Section 4). This can be done separately and at different times. When the main information is provided, the dedicated application is generated for the targeted smart device(s) and kept on this server. Any expert, including the family members and the user, can regenerate an application at any time if requirements change. Concerned people may be informed in that case.

At home, the user can require at any time an upload (ii) for a specific application, which is then stored in the ASIM Server. Finally, the user can use (iii) and take advantage of the new assistance application deployed on the ASIM Server.

3.5 The Hardware Box – ASIM Server

The whole system control is embedded in a hardware box named the ASIM Server or the SMM Server. The ASIM Server box is designed to facilitate the monitoring of medical data in connection with a hospital. The server supports health sensors such as non-invasive arterial pressure (PNI), oxygen saturation sensor (SatO₂), and electrocardiogram (ECG). These sensors are provided by Kaptalia Monitoring, which specializes in health sensors electronics.

The server box, shown in Figure 3, supports environment assistance through the Home Automation Control interface. Home Control involves many normative protocols, and the box has to be configured for multiprotocols to support the main standards. VITY chooses to implement support for KNX, 868 MHz, Bluetooth Low Energy (for the health sensors), and IP network.

Figure 3:

ASIM SMM Server Hardware Prototype Board.

Technically, the ASIM Server embeds a Freescale 1.2 GHz Cortex A8 processor with 512Mo DDR3 RAM. It is powered by PoE using the LAN 10/100 RJ45 connector; it comes with two USB 2.0 connectors for optional USB dongle; and a microSD slot card provides the mass storage needed. At system level, the server uses an Androïd 2.3 OS stored in the microSD card.

One interesting concept is that the card is multiprotocol enabled by embedding a Wi-Fi 802.11 g module, and can be extended by supporting extra protocols like 434 MHz and EnOcean by using optional USB keys. Thus, the hardware is interoperable with different standard protocols and can evolve to adapt to various scenarios.

4.1 Resolution Approach

The five issues identified in Section 1 need to be addressed. To resolve the first issue (i), the various responsibilities in the process, we chose to rely on model-driven engineering (MDE) to separate the points of view and establish the relationship of the model with the main acquisition concepts. Meta-models are then created while model transformations derive and aggregate the multisource information. This also ensures consistency for all models conform to meta-models.

To design an easily definable system (ii), we chose a questionnaire form. Thus, the generator proposes several pages grouped by topics and concerns that are significant for a family member, health-care personnel, or a technical installer.

This approach is also interesting for the process reconfiguration (iii) because page chaining can be modified and uncorrelated from meta-models data storage. Two processes have been created: one for the e-Health with hospital and one without a hospital link.

To be adaptive (iv), the UI aspect is completely generated from a skin pattern approach. Several skins are already defined in the system, so the user only has to select one. A skin is a set of a model transformation that gives the UI structure, images that provide the aspect part, and a renderer that translates the interface into the final applicative format.

To provide an upgradeable system (v), the equipment relies on a device database that supports external updates. The equipment defines not only the structural features but also the aspects for a specific target. This means that each device can expose configuration or control pages specific to itself, like a virtual equipment setup.

4.2 The Generative Workflow – Application Server

The proposed generative workflow presented in Figure 4 describes the main process in three phases: the Definition phase for the experts, the Junction phase mainly for the installer, and the Generation phase launched by the users or their family.

Figure 4:

The Generator’s Application Design Workflow.

The Definition phase needs to acquire the users’ specific concerns: their contacts, medical planning, and alerts (CNT model). Included will also be the home environment with all its equipment (ENV model) and interactions with the system, which include anything from basic services to complex scenarios (INT model). Finally, UIs with accessibility parameters (HCI model) are included.

The Junction phase describes how the technical part system will act physically. It is mainly the installer part as most parameters are technical. All the equipment is specified by defining the model and mark (or a generic type if necessary). Then, the low-level commands are detailed and coded in the output protocol format (KNX, IR, IP, etc.) for each equipment service. Originally, it was also intended to modify the final UI dynamically in WYSIWYG; however, this feature has not been implemented yet, due to time constraints.

Finally, the Generation phase, as it is named, aims at the generation of the complete final user control/command application for the SMM server. After this, the users can interact with the application as long as it is necessary, but they can make changes at any time. Changes can result from new software requirements, e.g., a user scenario that evolves, or the need to change a section part of their interactions, etc. Changes can also result from the hardware installation of a new piece of equipment. In these cases, a stakeholder or the users themselves (if it is possible) can modify the user application by restarting the design workflow anywhere. The state of the previous meta-models is saved so that one can reedit them completely or in part.

4.3 Generative Application – Questionnaire Form (UI)

The Generator Application has been built as a wizard questionnaire form, shown in Figure 5, to facilitate the models’ definition for the Definition and Junction phases.

Figure 5:

ASIM Generator Interface Using a Questionnaire Form on a Page.

The users must enter various parameters in successive pages. The supported features in the Definition phase for the e-Health with hospital scenario are as follows:

• A light patient profile: this part is designed to personify the system a little, but it is not the patient’s medical file.

• The medical measurement planning: this establishes the medical measures programmed for the patient. Alerts can be sent if a particular measure has not been entered after a defined delay.

• The medical alerts: these calibrate the measurement criteria.

Other parameters for e-Health without hospital are optional but necessary for home control and comfort usage, and other features are as follows:

• The home description: administrative information about the localization and postal address of the place of residence.

• The description of the rooms: this is necessary to establish space partitioning. Typically, the domicile is split into areas such as floors, which contain rooms.

• The installed home equipment: the system defines the list of equipment items per room. These are taken from a database maintained by VITY.

• Each equipment item has basic interaction services; they must be authorized or not depending on the user’s illness.

• The patient interacts with the system, thanks to basic services or global scenarios. Typically, scenarios roll out basic interactions or call up subscenarios. For example, at 7 a.m., the system automatically opens the shutters. Scenarios can also be grouped by modes.

• One of the most interesting features is the system’s ability to fire alerts. In the medical, energy, comfort, and security domains, alerts can be defined: at automatically programmed hours and days, or by system events, the system sends an alert and warns a close relative or a specific contact, and triggers a command or a scenario in reaction.

• Finally, the global interface is defined with the skin selection, the widgets grid size, and the architecture structure. From this, various targets can be created (e.g., smartphone, tablet, TV) with their own accessibility criteria.

The Generator application has been built as a Web application wizard, shown in Figure 5 to facilitate the application definition. The stakeholders must enter various parameters.

The flow enables a family member or a health-care provider to define four important concepts. The users’ profile with their contacts, medical planning, and the system alert rules that warn contacts are first defined in the contact model (CNT).

Then, the users’ environment model (ENV) is specified. It describes the home context with its rooms and the various controlled equipment for each room. At this stage, the family members/health-care personnel do not have to worry about the technical aspects such as the equipment bus protocol or address configuration.

The interaction model (INT) defines the way the users interact with their environment. The family members/health-care personnel can authorize or restrict the different equipment basic services. The interaction model also includes scenarios that build high-level services with advanced rules for a given functioning mode. A functioning mode is a user-requested system state that provides adapted services for that state. For example, when people are out of their home, they can activate the “I go out” mode that will close the shutters and turn off all the lights and electronic systems. Scenarios can be created for each mode; for example, in the “I am at home” mode, the “Wake-up” scenario can be designed to open shutters automatically at wake-up time.

Finally, the UI model (HCI) sets up the final UI depending on various accessibility parameters. By default, the system proposes four main interface assistants: a moving cursor for the selection of items, a speech description for blind persons, a voice recorder for command analysis, and a zoom for visually impaired persons. The interface model also defines the target screen size, the button size, and other accessibility parameters depending on a hardware target such as TV, smartphone, or tablet. The family members/health personnel can also choose between different skins to select a specific interface design.

An electric installer is included in the Junction phase. This phase mainly defines how the system works. The environment model has already been defined by the health personnel, but the equipment configuration needs to be set up. In this phase, the installer programs the type of equipment, the bus control protocol, equipment address, service IR recording, service address/command parameters, etc.

At the end, both the family members/health personnel and the installer can generate the application and export it on the SMM server. This enables the installer to check the global control system and the health personnel to activate the newly generated application.

5.1 Contact Meta-model (CNT)

The CNT model is presented in Figure 7. The central class is the Person; it contains an identifying field. This field is important to associate the medical measures to a hospital patient on the Hospital server. It also contains further information about the patient’s birthday and native language.

Figure 7:

CNT Meta-model Handling Profile, Contacts, Planning, and Alerts.

A person could have several Homes, but one principal home should probably be selected for ASIM. The Home class contains the postal address that supplements the structural definition in the ENV meta-model.

The Contacts field includes people who are necessary for monitoring purposes. A contact has a name, a preferred contact method (telephone, text message, email, etc.), contact details (phone number, email address), and a short description of the link with the patient (parent, neighbor, etc.). Contacts have their own time slot to define preferential communication slots.

The Day field is formatted as a set of seven days for a week. Each day set is encoded with one character (1=reachable, 0=unavailable), which gives a string text such as “1111100” to show that all weekdays are included but not the weekend. StartCallHour and EndCallHour define the time slot for the days concerned.

A patient could have several usage domains: medical, energy, comfort, and security. In each category, it is possible to create specific alerts. An alert rule details the resource that requires monitoring. The monitoring is based on several value ranges that define the inferior and superior thresholds for sending an alert. An alert has a priority level from 0: disable, 1: alert, 2: alarm, to 3: emergency. Each valid alert rule is linked to a basic or complex scenario action, and specific contact persons.

5.2 Environment Meta-model (ENV)

The ENV meta-model (Figure 8) and the INT meta-model are compatible with the ones proposed by the work done on Intel’Home [3]. In a synthetic way, the ENV model describes a habitation based on the component approach. Each floor or room is an Aggregate_Component that contains home automation components (Basic_Components). In ASIM, the meta-model has been enhanced to take into account generic components from the industrial database. In this way, a Generic_Component inherits from a Basic_Component to describe a real generic component driver because a Basic_Component is abstract. The components are now detailed using the type fields in the class Component, which means one can add new equipment without the drawbacks of modifying the meta-model to create the new classes.

Figure 8:

ENV Meta-model.

Each Generic_Component owns operations that are filled automatically by the VITY devices database (equipment). The operations are associated with the basic services provided by the equipment; for example, a HiFi station has services such as ON, OFF, VOL+, VOL-, etc.

The device abstraction part is handled as follows: the VITY company maintains the device database description the ASIM box can handle. The database is not as huge as one can imagine because the system includes abstract devices. Many devices use standard functions like ON/OFF for lights, adding a variable value for gradators; nevertheless, TV is more complex because it can support multiple functions and channels. For non-standard functionalities, dedicated devices are created to handle these specificities. We could also cite the Prosysts OSGi framework and OpenHAB, which are probably the closest solutions to provide an alternative way for the device abstraction layer.

One advantage of the VITY control device system is its learning feature that is able to record IR code commands without knowledge of the manufacturer protocol, and replay them on demand. The other way is the KNX standard, which provides KNX addresses to control compatible devices. These two concepts are the key to success in implementing the hardware abstraction layer.

5.3 Interaction Meta-model (INT)

The INT meta-model is close to the one described by Allegre et al. [3], with a few improvements (Figure 9). It defines the interactions the users/patients can have with the system. The Profile class contains interactions. In ASIM, it defines a specific usage mode that aims at grouping scenarios by usage. The basic interactions are nevertheless always at default index 0, and the other scenarios per mode are at index >0. The main differences from original studies are the forbidden interactions; these have not been implemented in ASIM, because interviews with Kerpape users have shown that patients are resistant to forbidden actions. This feature could be worth considering with wandering Alzheimer’s patients; however, this is another issue, beyond the scope of this paper.

Figure 9:

INT Meta-model.

As the components are integrated in the model as a resource together with their operations, resources and operations are automatically named with a conventional format: Room_Component_Operation. By convention in ASIM, resource at index 0 is reserved for an empty Resource (unassigned); similarly, an empty operation is at index 0. This is to avoid errors when saving the model, even if some interactions do not refer to a valid resource or operation.

For the interactions, we can find the services at the system control level, which is a Semantic_Level (/an action): either N1_BasicService for a basic service or N2_Composite for the scenario type. In the latter case, we rely on a state (in the finite state machine) to refer to another interaction. The composed services contain a basic service (N1) or a composed service (N2). It is important to define first the basic services at the default index 0, and then to create the scenarios that use basic or composed services of a different mode. Care must be taken to avoid a reentrant scenario, which is an infinite loop.

The activation types vary. In C1_Manual, the users can activate the order from their interface; in C2_SemiAutomatic, the users see a pop-up on the interface asking them to confirm the order. In the C3_Automatic activation type, the system automatically launches the order by respecting conditional rules: at a given schedule, or by a specific resource state.

5.4 UI Meta-model

The model-based UI community has been working on this particular topic since over 20 years. Calvary et al. [6] defined the Cameleon Reference Framework with four layers of abstraction for this purpose. Guerrero Garcia et al. [11] are working on the UsiXML language to implement such features, and [20] provide the MARIA language and development tool for similar goals.

The main problem with the UI is the variety of device combinations that require very custom-made UIs. Thus, combining multiple features into a UI might require complete UI redesigns for different devices, home settings, and modalities. It is not trivial to automate this process.

We do not resolve this issue, unless we adopt a complete mapping of all the cases. It is the worthy solution; nevertheless, the number of hardware UI targets could be grouped into three to four categories: small (mobile phone), medium (tablet or assistant), high (PC), and huge (4 K). Thus, we develop three different HCIs for each device; the UI definition is attached into the device database.

To perform this, the HCI model contains the global aspect data: skin type, interface organization, widget grid size, and all the accessibility parameters. This meta-model is adapted from previous studies carried out by Ferry on VisualSNI [10].

In Figure 10, the root SNI class embeds the global HCI data, while the UDCs define the target HCI material: TV, smartphone, or tablet. Each target includes other UDCs that detail the categories: audio, video, and control. Each category contains the input units used to acquire the ergonomic and accessibility parameters. The overall features implementation is covered by the ASIM kernel engine.

Figure 10:

SNI Meta-model.

5.5 VITY Meta-models

When the four models are defined, they are translated in the Junction phase to specific VITY target models. As transformations are defined at meta-model level, we have developed VITY meta-models. As they are proprietary, we cannot investigate too much; we can just observe that each device supports dedicated virtual equipment control pages per target. The UI model has a generic HCI description approach based on windows, pages, layers, and widgets. Then, this UI model is translated by using an elected skin and rendered to the target HCI platform format using the renderer.

7.1 Analyses of Work Performed

Currently, the application generator has been successfully developed. The software supports the acquisition of all the models: the patient environment, the definition of the home interactions, the global alerts and contacts system, etc.

A number of experiments were carried out with the generator, enabling us to obtain feedback through the electronic laboratory of the Kerpape Rehabilitation Center. These experiments provided assistance for the global process acquisition phase and many suggestions regarding graphical enhancements, always for the users’ benefits in terms of usability.

Next, we outline how the proposed system handles the five issues delineated in Section 1. The common multientry interface invites many individuals to interact in the definition phase, according to their skills. This is a crucial point, as multiple expert responsibilities are needed to define the system (i).

The acquisition phase has been successfully implemented with a questionnaire form. This helps structure the process while enabling the multiexpert support easily (ii). The use of MDE as the technical approach has enabled us to separate the concerns and establish relationships with the main acquisition concepts. The workflow itself is highly reconfigurable (iii), as new pages can be created, inserted, or modified to rebuild the definition process phase as needed.

The generator also embeds the application HCI skin engine, which enables the pattern-based generation and the customizable universal access parameters. It is responsible for the generation of all the control/command application parts in an adaptive (iv) and automated way. Thus, it is easy to regenerate an application based on the change of devices, new interface requirements, or health evolution.

With the generator, it has also been easy to add new equipment with a dedicated configuration page. For demonstration purposes, we have easily added two specialized TV components and a light system, mostly through a copy-and-modify approach. Thus, we can guarantee industry professionals that they can modify the system as they wish by entering new devices to extend their base. Hence, we provide a fully upgradeable system (v).

As a result, the solution handles interoperability in a convenient way, owing to an independent link between health-care meta-models and the industrial ones. Thus, the conceptual and the technical part are linked together and can be interoperable, actually with both the Danah academic platform and the ASIM industrial platform, opening the way for commercial deployment in a real-life e-Health context.

7.2 Perspectives for the Future

Experiments have shown that the Generator interface needs enhancements to be more user-friendly, usable, and intelligible. The questionnaire appears a good choice, but some pages need extra descriptions for an intuitive usage. For example, the title bar that shows the current position of an item in the form could be enhanced by using buttons that could also be used to jump to a specific page directly.

Thus, in the short term, it is possible to add a special interface to acquire automatic scenarios in which the users can enter the start-timed event. Similarly, the Contacts model provides privileged callback hour ranges, but the interface does not show the feature. The global generator aspect could be greatly improved, as our work focused mainly on the generator workflow and its feasibility itself and very little on the graphical part. A few studies in these areas ought to be envisaged.

In the medium term, the home acquisition could be enhanced by using a MagicPlan-like application. Additionally, a KNX importer can map KNX addresses directly for the installer. Complex scenarios use a linear description, but meta-models support both complex parallel and linear combinations; thus, a graphical editor with pluggable boxes could be developed.

In the long term, we could imagine a preview graphical editor where users can edit the final interface directly in real-time. Due to the Eclipse release used, the GEF-RAP was not ported on it. The next release (4.x) will now support GEF-RAP.

Future testing in medical structures would enable us to carry out experimentations with patients and obtain more data regarding the interest in this kind of system and the user improvements required.