Keywords

1 Introduction

In modern Japan, the development of a strong economy and technological progress have allowed most people to live comfortably. On the other hand, the number of people who experience stress in their daily lives is increasing to the extent that Japan has been labeled “a high-stress society.” In 2013, the Ministry of Health, Labor, and Welfare conducted a national workforce survey on “stress related to work and occupational life” and found that 52.3% of the respondents experienced strong feelings of stress. In 2018, the same survey revealed that the proportion of respondents experiencing feelings of stress had increased to 58% [1]. Research on emotions is important to support people who are suffering in this stressful society. There have been many studies on the classification of emotions. Emotions are said to have various effects on the human body, and research has mainly been conducted using biological information. There have been many studies based on biological information, such as Electroencephalogram (EEG), heart rate variability, and respiration [2, 3]. However, many of these measurements were taken while the subjects were resting; there have been few studies in which these measurements were taken while the subjects were in motion. In addition, research in which biological devices are used to collect information can be problematic as subjects find it burdensome to wear these devices.

Venture et al. concluded that “walking can identify human emotions” [4]. Their investigation used a human-like model to simulate five different emotions while walking. When specific parameters, such as speed and posture, were changed, the emotions perceived by observers changed accordingly. The observers’ average accuracy rate in recognizing the four different emotions from the gait of the model was 78%. This suggests the possibility of extracting individual characteristics from walking styles and numerically predicting human emotions. Toshimitsu Musha proposed an emotional spectrum analysis method (ESAM) as a method for numerically analyzing and measuring the human mind [5]. This is a method that quantifies emotional states using three determinants and classifies them into four distinct emotions. He showed the effect of music therapy on the psychological states of subjects numerically by measuring their brain waves. Utsunomiya et al. and Takeuchi et al. have used similar approaches in their studies [6, 7].

Walking refers to the relatively slow movement of an animal’s legs and is distinguished from “running,” which denotes the movement of legs at high speed. Although it is said that emotions can be discerned in people’s facial expressions and the quality of their voice, the studies described above show that emotions are also manifested by walking. To our knowledge, there has been no research on the classification of emotions through analysis of walking movements, so it can be said that the classification of emotions according to walking styles is novel and useful.

Gait refers to the state of a person walking, and typical examples include stride length and speed. The gaits used in this study and their definitions are presented in Table 1.

Table 1. Gaits and their definitions
Full size table

The purpose of this study was to classify walking motions into two emotions (positive and negative) by linear analysis with reference to ESAM. Table 2 presents a comparison with previous studies.

Table 2. Comparison with previous research
Full size table

2 Method of Analysis

2.1 Emotional Spectrum Analysis Method

The ESAM is a method of numerically analyzing emotions by using the linear models in Eqs. 1 and 2. A is defined as a “sensitivity matrix” for classifying emotions; Z is defined as a “sensitivity vector,” representing subjective emotional values; and C is defined as an “input vector,” representing measured data. The sensitivity matrix converts the input vector into the sensitivity vector.

$$ Z = AC $$
(1)
(2)

2.2 Emotion Analysis Method (This Study)

Emotion Vector: Z.

The emotion vector evaluates the subjective value of the subject numerically and uses it for the determinant. Therefore, in this study, the input value of the emotion vector was determined using a questionnaire.

Biological Vector: C.

The biological vector determines the input value based on walking measurement data. In this study, we used five gaits: stride, arm swing, speed, heel height, and wrist height. The reason for choosing these gaits is that the values are generally expected to be large for positive emotions and small for negative emotions.

Emotion Matrix: A.

The emotion matrix is a matrix calculated from Eq. 3 transformed from Eq. 1. This matrix was calculated using the emotion vector and the inverse matrix of the biological vector.

$$ A = ZC^{ - 1} $$
(3)

Estimated Emotion Vector: \( \widehat{\text{Z}} \).

The matrix used for emotion classification was defined as the “estimated emotion vector” and is represented by \( \widehat{\text{Z}} \). This matrix represents the subject’s estimated subjective value. As with the emotion vector, it was calculated using Eq. 4.

$$ \widehat{Z} = AC $$
(4)

Classification Method.

The emotion classification was performed by comparing the emotion vector with the estimated emotion vector. If \( z_{i} \) in Eq. 5 was the same as \( \widehat{{z_{i} }} \) in Eq. 6, the classification was correct, but if the signs were different, the classification was incorrect.

$$ Z = \left( {z_{1} \quad z_{2} \quad z_{3} \quad z_{4} \quad z_{5} } \right) $$
(5)
$$ \widehat{Z} = \left( {\widehat{{z_{1} }}\quad \widehat{{z_{2} }}\quad \widehat{{z_{3} }}\quad \widehat{{z_{4} }}\quad \widehat{{z_{5} }}} \right) $$
(6)

3 Experiment

3.1 Overview

Emotion Matrix Creation Experiment.

We performed an emotion matrix creation experiment as a preliminary experiment for this study. In this experiment, an emotion matrix was created from an emotion vector whose input value was determined from the questionnaire results and a biological vector whose input value was determined by data on walking.

Estimated Emotion Vector Creation Experiment.

We conducted an estimated emotion vector creation experiment as the main experiment in this research. The estimated emotion vector was created from the emotion matrix created in Sect. 3.2.1 and the biological vector whose input value was determined by the newly measured data from the walking experiments. Emotion classification was performed by comparing the results with the emotion vector determined from the actual questionnaire results.

3.2 Emotion Analysis Method

Input Value of the Emotion Vector.

In this study, the input value was determined by a questionnaire. Subjects were stimulated by classical music during the experiment. There were two types of questionnaire, one with a 7-point rating and another with a 100-point rating. These questionnaires were intended to make it easier for the subjects to make more accurate evaluations and were not used directly in data processing. The subject answered with a 7-point scale and then answered with a 100-point scale. This questionnaire allowed responses with a value between −100 and +100, providing a more detailed evaluation of the subject’s emotions. A value closer to −100 indicated a negative tendency, and a value closer to +100 indicated a positive tendency. The emotion vector was converted from −1 to +1.

Input Value of the Biological Vector.

The biological vector represents the data from the measurements of walking. In this study, we used five gaits: stride, arm swing, speed, heel height, and wrist height. The stride is the maximum value of the distance between the two heels, the arm amplitude is the maximum value of the distance between the two wrists, the speed is the speed when walking at 5 m, and the foot height is the maximum value of the height of the heels from the ground. The wrist height is the maximum value of the height of both wrists from the ground. The stride, arm swing, foot height, and wrist height are the average of all local maxima measured multiple times in a single walk. Therefore, the definition of each element is as presented in Eq. 7–Eq. 11.

$$ stride[i] = \frac{1}{n}\sum\limits_{k = 1}^{n} {stride\_peak(k)} $$
(7)
$$ arm\;swing[i] = \frac{1}{n}\sum\limits_{k = 1}^{n} {arm\;swing\_peak(k)} $$
(8)
$$ speed[i] = 5/t(i) $$
(9)
$$ foot\;height[i] = \frac{1}{n}\sum\limits_{k = 1}^{n} {foot\;height\_peak(k)} $$
(10)
$$ hand\;height[i] = \frac{1}{n}\sum\limits_{k = 1}^{n} {hand\;height\_peak(k)} $$
(11)

Equation 7 is the definition of the stride. stride[i] is the stride of the i-th set, and stride_peak(n) is the n-th maximum. Equation 8 is the definition of the arm swing. arm swing[i] is the arm swing of the i-th set, and arm swing_peak(n) is the n-th maximum. Equation 9 is the definition of speed. speed[i] is the speed of the i-th set, and t (i) is the time taken for the i-th set walking. Equation 10 is the definition of heel height. foot height [i] is the heel height of the i-th set, and heel height_peak (n) is the n-th maximum. Equation 11 is the definition of wrist height. wrist height [i] is the height of the wrist in the i-th set, and wrist height_peak (n) is the n-th maximum. In order to equalize the effect of each gait, normalization was performed for each gait so that the maximum value was 1 and the minimum value was 0.

Calculation of Emotion Analysis Method.

The determinant in this study was the number of rows and columns presented in Fig. 1. In the biological vector, \( c_{1n} \) is the stride of the n-th set, \( c_{2n} \) is the arm swing of the n-th set, \( c_{3n} \) is the speed of the n-th set, \( n_{4n} \) is the foot height of the n-th set, and \( c_{5n} \) is the wrist height of the n-th set. In the emotion vector, \( Z_{n} \) indicates the n-th set of the questionnaire results.

Fig. 1.
figure 1

Calculation of the emotion analysis method

Full size image

3.3 Subject

The subjects were five healthy men in their 20 s. The subjects were anonymized and referred to as Subs. 1–5. Subjects wore bodysuits with trackers as shown in Fig. 2.

Fig. 2.
figure 2

Bodysuits

Full size image

3.4 Stimulation Method

We chose classical music to stimulate the subjects’ emotions. This is because listening to music has been shown to change emotions [8]. In addition, stimulation of other senses (sight, taste, and touch) was difficult to achieve in walking subjects, and using the sense of smell was not appropriate from the viewpoint of handling in an experimental studio.

3.5 Experiment Flow

First, the subject put on a bodysuit and waited with headphones on at the designated starting point. After that, he listened to classical music through the headphones for 10 s. Then, we walked along the marked-out 7-m-long section (with a 5-m-long section in the center), and finally, we let the subjects answer the questionnaire. Taking the above process as one set, five sets each of the emotion matrix creation experiment and estimated emotion vector creation experiment were undertaken, each for a total of 10 sets.

4 Results of the Analysis and Classification

Table 3 presents a comparison between the estimated emotion vector created in the emotion matrix creation experiment and the biological vector representing the newly measured data from the walking experiments and the emotion vector representing the actual questionnaire result. The emotion classification accuracy was 20% for Sub. 1, 40% for Sub. 2, 20% for Sub. 3, 20% for Sub. 4, and 80% for Sub. 5.

Table 3. Comparison of Z and \( \widehat{\text{Z}} \) (Gray matches)
Full size table

5 Discussion

5.1 Discussion of the Rate of Correct Classification

The emotion classification accuracy rate of Sub. 5 was 80%. This rate was higher than the other four subjects. Figure 3 presents the walking data. The vertical axis represents the data from the walking experiments after normalization. The horizontal axis represents the number of sets and recalled emotions.

Fig. 3.
figure 3

Walking data (Sub. 5)

Full size image

In the emotion matrix creation experiment, in the second set that reminded the subject of positive, the stride was 0.96, the arm swing was 0.65, the speed was 0.86, the heel height was 1.00, and the wrist height was 0.84. Similarly, in the third set that reminded the subject of positive, the stride was 1.00, the arm swing was 0.79, the speed was 1.00, the heel height was 0.69, and the wrist height was 1.00. On the other hand, in the fourth set that reminded the subject of negative, the stride was 0.23, the arm swing was 0.00, the speed was 0.00, the heel height was 0.23, and the wrist height was 0.23. Similarly, in the fifth set that reminded the subject of negative, the stride was 0.00, the arm swing was 0.12, the speed was 0.22, the heel height was 0.24, and the wrist height was 0.00. From the above, for Sub. 5, the values from the walking experiments were large if the recalled emotion was positive and small if the emotion was negative. In other words, the data from the walking experiments from Sub. 5 were expressing emotion.

In the estimated emotion vector creation experiment, the same tendency was observed, except in the eighth set. The 6th, 7th, 9th, and 10th sets in which emotions were expressed in the data from the walking experiments were correctly classified. On the other hand, the eighth set, where emotions were not expressed in the data from the walking experiments, was not correctly classified.

From the above, we consider that Sub. 5 recorded a high classification accuracy rate because the emotion tended to be expressed in the data from the walking experiments.

5.2 Discussion for Misclassification

The emotion classification accuracy rate for Sub. 1, Sub. 3, and Sub. 4 was 20%, and for Sub. 2, it was 40%. All rates were lower than for Sub. 5. Due to space limitations, only the results for Sub. 3 are presented in Fig. 4.

Fig. 4.
figure 4

Walking data (Sub. 3)

Full size image

In both the emotion vector creation experiment and the estimated emotion vector creation experiment, the data from the walking experiments were scattered regardless of the recalled emotion. From the above, we consider that Sub. 3 recorded a low accuracy rate in the classification of emotions because emotion was not expressed in the data from the walking experiments. Subs. 1, 2, and 4 also had the same tendency as Sub. 5.

The emotion classification accuracy rate for Sub. 1, Sub. 3, and Sub. 4 was 20%, and for Sub. 2, it was 40%. All rates were lower than for Sub. 5. Due to space limitations, only the results for Sub. 3 are presented in Fig. 4.

In both the emotion vector creation experiment and the estimated emotion vector creation experiment, the data from the walking experiments were scattered regardless of the recalled emotion. From the above, we consider that Sub. 3 recorded a low accuracy rate in the classification of emotions because emotion was not expressed in the data from the walking experiments. Subs. 1, 2, and 4 also had the same tendency as Sub. 5.

6 Conclusion

In this study, we conducted two experiments with the purpose of numerically analyzing the act of walking and classifying emotions: an emotion matrix creation experiment and an estimated emotion vector creation experiment. There were large differences of up to 60% between the subjects in their emotion classification accuracy rates. On analyzing the data, it was found that subjects whose emotions were easily expressed in the data from the walking experiments had a high emotion classification accuracy rate, but those who did not easily express emotions had a low emotion classification accuracy rate.

For the future, it is necessary to propose a new method of analysis and improve it by taking into account gait characteristics for different subjects. In addition, we classified emotions into two categories, positive and negative, but it is also necessary to classify emotions in more subtle ways (surprise, fear, etc.), which would be useful for further research. In this study, we focused on five gaits: stride, arm swing, speed, heel height, and wrist height.