DE102015217335A1 - Method for determining a classifier for determining states based on electroencephalography data - Google Patents

Abstract

A method (300) for determining a state classifier is described. The method (300) includes determining (304) a plurality of training records. The method (300) further comprises determining (305, 306), based on the plurality of training data sets, weighting values for individual originating features, wherein a weighting value for an originating feature comprises a relevance of this originating characteristic for the status Classifier, and where the weighting values are determined by reducing or increasing an optimization function. The optimization function includes a first elimination term that rewards it if all the source features of a particular channel (405) of a plurality of different channels (405) of an electroencephalograph (105) are irrelevant to the state classifier. Furthermore, the optimization function includes a second elimination term that rewards it when a single origin feature is irrelevant to the state classifier. It is thus possible on the basis of the weighting values to select a relevant subset of the plurality of origin features for the state classifier.

Description

The invention relates to a method and a corresponding control unit for determining a state classifier for determining the state of a person, in particular a driver of a motor vehicle.

A motor vehicle, in particular a road motor vehicle, already typically comprises a large number of driver assistance systems (such as a lane departure warning system, an automatic cruise control (ACC), etc.), which assist a driver of the motor vehicle in guiding the motor vehicle. In order to increase the quality of assistance and / or comfort of a driver assistance system (FAS), the functionality of a FAS may vary from a driver's condition, e.g. B. of a degree of attention of the driver to be made dependent. For example, a warning signal generated by the FAS may be generated depending on the driver's condition (eg, relatively early or relatively late).

In this connection, it is necessary to reliably determine the condition of a person, particularly a driver of a vehicle. In this case, sensor data of a plurality of channels or electrodes of an electroencephalograph (EEG) can be evaluated. The present document deals with the technical task of determining the condition of a person, in particular a driver of a vehicle, in a reliable and efficient manner on the basis of the sensor data of the channels of an EEG. In particular, the present document addresses the technical problem of providing a state classifier that can reliably and efficiently determine a person's condition based on EEG sensor data.

The object is solved by the independent claims. Advantageous embodiments are u. a. in the dependent claims.

In one aspect, a method of determining a state classifier is described. The method may be performed by a microprocessor (eg, by a microprocessor in a vehicle). The state classifier is set up to assign a first state of a reduced feature vector to a first state of a person, in particular a driver of a vehicle (eg a road motor vehicle). In particular, the state classifier may be configured to determine the current state of a driver of a vehicle during a journey. For this purpose, a current (first) value of the reduced feature vector can be determined on the basis of current sensor data. The state classifier is then configured to assign a current (first) state of the driver to this current (first) value of the reduced feature vector. Thus, a current state of the driver can be determined in a reliable and effective manner.

The state classifier is typically set up to select a state of a person from a plurality of predefined states. The different predefined states can also be considered as different classes. The set of different predefined states is typically limited to a relatively small number (eg, 10, 5, 4, 3 or less) of states. A state of a person may, for. B. indicate a degree of attention of the person.

The reduced feature vector typically includes a plurality of different features that may be determined based on sensor data from different channels of an electroencephalograph (EEG). An EEG comprises a plurality of different electrodes (typically 10 electrodes or more) which are located at different positions on a person's head, e.g. As a driver of a vehicle, can be attached to detect brain waves at the different positions as sensor data. The different electrodes may be referred to as different channels of an EEG. Sensor data (i.e., electrode signals) are provided for each channel of the EEG.

Typically, based on the sensor data, a relatively large amount of different features may be calculated (also referred to herein as originating features). In particular, multiple (typically corresponding) origin features may be determined for each channel of the plurality of channels. Exemplary source features which can be determined on the basis of the sensor data of a channel of an EEG are: a spectrum of a temporal section of the sensor data of a channel; an entropy of a temporal section of the sensor data of a channel; and / or a fractal dimension of a temporal section of the sensor data of a channel.

The reduced feature vector includes only a subset of the relatively large set of source features that can be calculated based on the sensor data. The method described in this document is set up to determine this subset such that even with a reduced feature vector (with a reduced set of features), a state classifier with a high classification quality can be provided. At the same time, by reducing the amount of features, the computational complexity of the state classifier is reduced, so that a state classifier can be provided that can be used in real time (eg, on a control device of a vehicle).

The method includes providing training data that includes sensor data of the plurality of different channels of an EEG. The method further includes determining, based on the training data, a plurality of training records. In this case, a training record comprises a value of an origin feature vector and a corresponding actual state of the person. The source feature vector includes a plurality of source features, wherein the plurality of source features include at least one feature for each channel of the plurality of different channels. In particular, the plurality of origin features for at least one channel of the plurality of different channels (typically for all channels of the plurality of different channels) may include a plurality of origin features.

In addition, the method includes determining, based on the plurality of training records, weighting values for the individual source features. The weighting values typically include at least one weighting value for each originating feature of the plurality of originating features. Furthermore, the weighting values may include at least one weighting value for each possible state of the plurality of predefined states. The weighting values may be summarized in a weighting matrix.

A weighting value for an origin characteristic indicates a relevance of this origin characteristic for the state classifier. For example, an increasing (absolute) value of a weighting value may indicate an increasing relevance of the corresponding source feature to the state classifier. The weighting values thus make it possible to make a selection as to which origin features are relevant to the state classifier (and thus should be included in the reduced feature vector) and which origin features are irrelevant to the state classifier (and thus, without causing a substantial reduction of the classification quality, can be disregarded in the state classifier).

The weighting values are determined by reducing or increasing (or optimizing) an optimization function. For this purpose, z. For example, a gradient method may be used to minimize or maximize the optimization function. The optimization function includes a first elimination term that rewards it if all of the source characteristics of a particular channel of the plurality of different channels of the EEG are irrelevant to the state classifier. In particular, the first elimination term may provide an incentive for either all weighting values of the source features of a particular channel to indicate irrelevancy of the corresponding source features (eg, all weighting values are zero), or all weighting values of the source Features of a particular channel indicate a relevance of the corresponding source features (eg, that all weighting values are substantially different from zero). The first elimination term can thus be an incentive to completely eliminate individual channels, and thus not to be considered in the state classifier.

Furthermore, the optimization function includes a second elimination term that rewards it when a single origin feature is irrelevant to the state classifier. In particular, the second elimination term may provide an incentive for either all weighting values of a particular source feature to indicate irrelevancy of the corresponding source feature (eg, all weighting values are zero), or all weighting values of this source feature to be one Show relevancy of the source feature (for example, that all weighting values are substantially different from zero). The second elimination term can thus be an incentive to uniquely eliminate individual origin features and thus not to be considered in the state classifier.

The elimination terms may also be referred to as population terms because they affect the population of a weighting matrix having the weighting values. The first elimination term causes a reduction of the occupation of the weighting matrix for the total time of the original Characteristics of individual channels. The second elimination term causes a reduction in the population of the weighting matrix for individual origin features.

The optimization function thus enables a simultaneous channel selection and feature selection in one optimization step. Thus, a reduced feature vector can be provided which enables a high classification quality with low computational complexity.

The method may further comprise selecting, based on the weighting values, a subset of the plurality of source features for a reduced feature vector. For this purpose, a comparison of the weighting values (or representative weighting values determined therefrom) may be performed with a weighting threshold.

The method may further include providing the state classifier based on the weighting values for the selected subset of source features. In particular, the weighting values for the selected subset of source features may be used to determine classifier values by which the first value of the reduced feature vector may be multiplied to determine the state of the person. The classifier values may optionally correspond to the weighting values for the selected subset of originating features.

The optimization function typically includes an error term indicating an average deviation of predicted states of the person from actual states of the person. In this case, the error term can determine a predicted state of the person for a training dataset by applying (eg by multiplication) at least one weighting value to the value of the origin feature vector of this training dataset.

In particular, the error term can be determined on the basis of the following formula J = 1 / 2∥X ^ W - Y∥ 2 / F In this case, W corresponds to a weighting matrix which comprises the weighting values. X ^ is a training data matrix that includes the values of the source feature vector from the plurality of training records. Y is a state matrix indicating the actual states of the person from the plurality of training records. Furthermore, ∥ ∥ _F corresponds to a Frobenius norm.

Determining weighting values includes determining the first elimination term and determining the second elimination term. The determining of the first elimination term may include determining a plurality of channel norms for the corresponding plurality of channels, wherein a channel standard for a channel comprises an L2 norm of the weighting values of the one or more origin features of that channel can. Furthermore, determining the first elimination term may include determining an L1 norm of the plurality of channel norms. The combination of an L2 standard with an L1 standard can result in particularly effective elimination of individual channels.

wobei W die Gewichtungsmatrix ist, wobei M der Anzahl von unterschiedlichen Kanälen entspricht, wobei Quelle_m der Menge der Ursprungs-Merkmale eines m-ten Kanals entspricht, und wobei w_q eine Untermatrix der Gewichtungsmatrix ist, die nur die Gewichtungswerte der ein oder mehreren Ursprungs-Merkmale für einen bestimmten m-te Kanal umfasst.In particular, the first elimination term can be determined based on the following formula

where W is the weighting matrix, where M is the number of distinct channels, where source _{m is} the set of origin features of an mth channel, and _{wq is} a sub-matrix of the weighting matrix containing only the weighting values of one or more origin Features for a particular mth channel.

Determining the second elimination term may include determining a plurality of feature norms for the corresponding plurality of origin features, wherein a feature standard for an origin characteristic is an L2 norm of the one or more weight values for that originating feature. Feature includes. Furthermore, determining the second elimination term may include determining an L1 norm of the plurality of feature norms. The combination of an L2 standard with an L1 standard can result in a particularly effective elimination of individual features.

In particular, the second elimination term can be determined based on the following formula ∥W∥ _1,2 = Σ _d ∥w _d ∥ ₂ where w _{d is} a sub-matrix of the weighting matrix that includes only the weighting values for a particular origin feature.

The first elimination term and the second elimination term may be considered via respective factors C ₁ and C ₂ (also referred to as hyperparameters) in the optimization function. The method may further comprise determining values for the factors C ₁ and C ₂ such that a classification grade of the determined state classifier is increased (possibly maximized). This can be z. B. be made possible by an iterative method in which different state classifiers are determined based on different values of the factors C ₁ and C ₂ . The state classifier with the relatively highest classification quality can then be selected. Thus, the quality of the state classifier can be further increased.

In another aspect, a control unit (eg, a control device for a vehicle) configured to execute the method described in this document is described. For example, the method may be used during operation of a vehicle to adapt the state classifier to a driver of the vehicle. Thus, the determination of the condition of the driver can be further improved.

According to a further aspect, a vehicle (in particular a road motor vehicle, for example a passenger car, a truck or a motorcycle) is described, which comprises the control unit described in this document.

In another aspect, a software (SW) program is described. The SW program may be set up to run on a processor (eg, on a controller) and thereby perform the method described in this document.

In another aspect, a storage medium is described. The storage medium may include a SW program that is set up to run on a processor and thereby perform the method described in this document.

It should be understood that the methods, devices and systems described herein may be used alone as well as in combination with other methods, devices and systems described in this document. Furthermore, any aspects of the methods, devices, and systems described herein may be combined in a variety of ways. In particular, the features of the claims can be combined in a variety of ways.

Furthermore, the invention will be described in more detail with reference to exemplary embodiments. Show

1 a block diagram with exemplary components of a vehicle;

2 an exemplary extraction of a feature from sensor data;

3 a flowchart of an exemplary method for determining a classifier for determining the state of a driver of a vehicle; and

4a . 4b and 4c exemplary arrangements of electrodes of an electroencephalograph.

As set forth above, the present document is concerned with the reliable and efficient determination of the condition of the driver of a vehicle. In this context shows 1 a block diagram of an exemplary vehicle 100 , The vehicle 100 includes a driver's position 102 on which a driver of the vehicle 100 can sit to the vehicle 100 respectively. The vehicle 100 can be a variety of different sensors 103 . 104 . 105 which are set up sensor data 111 with respect to the driver and to a control unit 101 forward.

An exemplary sensor 103 . 104 . 105 is a picture camera 103 (For example, a stereo camera), which is set, image data related to the driver as sensor data 111 provide. In particular, an image camera 103 be set up to capture a head of the driver. Another exemplary sensor 103 . 104 . 105 is an eye tracker 104 which is adapted to image capture at least one eye of the driver and image data relating to the eye of the driver as sensor data 111 provide. Furthermore, an electrocardiograph and / or an electroencephalograph 105 at the seating position 102 be arranged by the driver to z. B. Sensor data 111 with respect to the heart rhythm and / or with respect to the brain's flow of the driver.

The sensor data 111 can from the control unit 101 be evaluated to determine the condition of the driver. For this purpose, it may be possible to access stored data (eg to a predetermined state classifier) from a memory unit 109 be accessed. In particular, the sensor data 111 be assigned a state of the driver by means of a predetermined state classifier. Furthermore, if appropriate, a probability can be determined with which the driver has the determined state. Exemplary state classifiers are a Bayes classifier, an Expectation Maximization classifier, a K-Means classifier, and / or a Support Vector Machine classifier.

A state classifier is typically arranged to associate a feature (also referred to herein as a value) of a feature vector x _i at time i, a class y _i at time i. The expression of the feature vector x _i at the time i can be based on the sensor data 111 be determined at or about the time i. In particular, based on the sensor data 111 the expressions for a plurality of different features x _di , where d = 1, ..., D, are determined, where D is the total number of features determined. The expression of the feature vector x _i results from the occurrences of the plurality of features x _di as x _i = [x _1i ,..., X _Di ] ^T.

• • Ein Deskriptor der, durch eine Bildkamera 103 erfassten, Bilddaten. Dabei kann der Deskriptor z. B. gemäß einem HOG(Histogram of Oriented Gradients)-Modell, einem LBP(Local Binary Patterns)-Modell, und/oder einem Wavelet-Modell ermittelt werden. Der Deskriptorwert für einen bestimmten Satz von Bilddaten stellt dabei eine Ausprägung des Deskriptors dar.
• • Eine Blinkrate, eine Blickrichtung, etc., die auf Basis der Sensordaten 111 eines Eyetrackers 104 ermittelt werden können. Ein bestimmter Wert der Blinkrate und/oder ein bestimmter Blickwinkel stellen dabei eine Ausprägung dieser Merkmale dar.
• • Eine Herzfrequenz, eine Varianz der Herzfrequenz, etc., die auf Basis der Sensordaten 111 eines Elektrokardiographen 105 ermittelt werden können. Ein bestimmter Wert der Herzfrequenz und/oder der Varianz stellen dabei Ausprägungen dieser Merkmale dar.
• • Die Energie in einem bestimmten Frequenzbandes eines EEG-Kanals, die Entropie in einem EEG-Kanal, etc., die auf Basis der Sensordaten 111 eines Elektroenzephalographen 105 ermittelt werden. Die auf Basis der Sensordaten ermittelten Werte dieser Merkmale stellen dabei Ausprägungen dieser Merkmale dar.

Exemplary features are:

• • A descriptor of, through a picture camera 103 captured, image data. In this case, the descriptor z. B. according to a HOG (Histogram of Oriented Gradients) model, a LBP (Local Binary Patterns) model, and / or a wavelet model are determined. The descriptor value for a specific set of image data represents an expression of the descriptor.
• • A blinking rate, a viewing direction, etc. based on the sensor data 111 an eyetracker 104 can be determined. A specific value of the blink rate and / or a specific angle of view represent an expression of these features.
• • A heart rate, a variance of the heart rate, etc. based on the sensor data 111 an electrocardiograph 105 can be determined. A specific value of the heart rate and / or the variance represent characteristics of these features.
• • The energy in a given frequency band of an EEG channel, the entropy in an EEG channel, etc., based on the sensor data 111 an electroencephalograph 105 be determined. The values of these features determined on the basis of the sensor data represent characteristics of these features.

2 Illustratively illustrates the determination of an expression of a feature 203 based on the sensor data 111 a sensor 103 . 104 . 105 , The sensor data 111 typically include a temporal sequence of samples 201 (or readings) that are at a corresponding sequence of times 204 was recorded. Typically, a window 202 of samples 201 from the sequence of samples 201 considered to be the characteristic of a feature 203 at a certain time 204 to investigate. The window 202 can do it like in 2 represent a specific time segment before a current time 204 include. The window 202 can be gradually shifted to a sequence of occurrences of the feature 203 for a corresponding sequence of times 204 to investigate.

It can thus be based on the sensor data 111 at any time i 204 an expression of a feature vector x _i with a variety of features 203 be determined. By means of a state classifier (ie by means of a predefined assignment function), this characteristic of the feature vector x _{i can be assigned} a class y _i from a multiplicity of predetermined classes {1,..., K}, where K is the number of predetermined classes , The different classes may indicate different states (eg, a different degree of attention) of the driver.

The determination of a multiplicity of different feature variants for a multiplicity of different features 203 based on sensor data 111 a plurality of different sensors or data sources 103 . 104 . 105 can be very compute-intensive. In particular, the required computation may not be possible by the processor of a control unit 101 of a vehicle 100 delivered in real time become. To have a real-time application in a vehicle 100 Therefore, the number D of features should be allowed 203 be limited. It then raises the question of what features 203 should be selected even with a limited number D of features 203 be able to determine the condition of the driver in the most precise way possible. In particular, the question arises what characteristics 203 and which sensors (ie data sources) 103 . 104 . 105 should be used to determine the condition of the driver. In other words, it should be the best possible combination of features 203 and data sources 103 . 104 . 105 be selected for the determination of state. The following describes a method by which, at the same time, the features in an efficient and precise manner 203 and the underlying data sources 103 . 104 . 105 can be determined for a reliable state classifier.

wobei {(x_i, y_i)} N / i=1 gelabelte Trainings-Datensätze sind, bei denen eine tatsächliche Ausprägung eines Merkmalsvektors x_i einer tatsächlichen Zustandsklasse y_i des Fahrers zugeordnet ist. Die Ausprägungen des Merkmalsvektors x_i werden in der Matrix X ^ (die auch als Trainingsdatenmatrix bezeichnet wird) zusammengefasst, so dass jede Zeile der Matrix X ^ eine Ausprägung des Merkmalsvektors x_i umfasst und so dass jede Spalte der Matrix X ^ ein unterschiedliches Merkmal x_di umfasst. Die Matrix X ^ ist somit eine N×D Matrix.In particular, a method is described, with the same features 203 and the data sources 103 . 104 . 105 can be selected such that a mean (square) error measure for the classification is reduced (in particular minimized). For this purpose, a (linear) discriminant analysis (English, Linear Discriminant Analysis) is used, with the following optimization problem for the selection of features 203 can be based on:

in which {(x _i , y _i )} N / i = 1 labeled training data sets, in which an actual occurrence of a feature vector x _i of an actual state of class y _i is assigned to the driver. The occurrences of the feature vector x _i are summarized in the matrix X 1 (which is also referred to as the training data matrix), so that each row of the matrix X 1 comprises an expression of the feature vector x _i and so that each column of the matrix X 1 has a different feature x _di includes. The matrix X ^ is thus an N × D matrix.

wobei n_j die Anzahl von Trainings-Datensätzen mit der Zustandsklasse j ist, und wobei N der Gesamtzahl der Traings-Datensätze entspricht.The matrix Y (also called the state matrix) is an N × K matrix that displays the condition classes for the different occurrences of the feature vector. The individual entries Y _{ij of} the matrix Y can be determined as follows

where n _{j is} the number of training records having the class of condition j, and where N is the total number of training records.

The matrix W is a D × K matrix that is in a row, K weighting values w _dk for a given feature 203 displays. The weighting value w _dk indicates how relevant a particular feature d 203 for recognizing a particular condition class k.

∥ ∥ _F corresponds to the Frobenius norm. Although it can be shown that the above optimization problem is a problem that reduces (in particular minimizes) a mean square error, it is nevertheless optimal with respect to a mean square discriminant analysis. On the basis of the above-mentioned optimization problem (in particular minimization problem), weighting values w _dk can therefore be used for the individual features 203 to determine the relevance of each feature 203 for a state classifier. However, in practice, based on the weighting values w _dk , it is typically not possible to make clear decisions about which features 203 from which data sources 103 . 104 . 105 should be considered in a state classifier. In particular, the determined weighting values w _dk typically do not permit a clear separation into a first group of relevant features 203 and a second group of non-relevant characteristics 203 , This is because the determined matrix W is typically not sparse, ie, the matrix W typically has no or only a few weighting values w _dk that are (nearly) zero, or that are substantially smaller than other weighting values ,

The optimization problem can be supplemented by one or more further terms by which the occupancy rate of the matrix W can be influenced. In particular, one or more terms which reward it when the determined matrix W is weakly occupied. Preferably, a first population term is used, a weak population with respect to the set of features 203 a specific data source 103 . 104 . 105 to reward. Furthermore, a second population term can be used, a weak population in terms of individual characteristics 203 to reward (regardless of which data source 103 . 104 . 105 the respective characteristics 203 based).

The first staffing term (to select individual data sources 103 . 104 . 105 ) can be formulated as follows: J ₁ = C ₁ ∥W∥ _{source, 2} where C _{1 is} a hyperparameter by which a focus on the first population term, relative to other terms of the optimization problem, can be changed. The standard ∥W∥ _{source, 2 is} preferably defined as follows:

This standard determines the root mean square of the weighting values w _dk for all features q of a given data source m (ie for all q ε source _m ) and then the sum of these quadratic means over all M data sources, ie for m = 1, ..., M Where W ^{m is} a sub-matrix of the weighting matrix W which comprises the lines of the weighting values w _dk for the features q of the data source m. It is true that the number D of all features considered 203 , the number M of the considered data sources 103 . 104 . 105 multiplied by the number Q _{m of} the features 203 the individual data sources considered 103 . 104 . 105 corresponds, with m = 1, ..., M.

The first population term can be reduced if (not including the first population term) the weighting values w _{dk of} the features 203 a first data source 103 . 104 . 105 would tend to be small and the weighting values w _{dk of} the features 203 a second data source 103 . 104 . 105 would tend to be large, all weighting values w _{dk of} the features 203 the first data source 103 . 104 . 105 set to zero. The first population term thus causes a selection of relevant data sources 103 . 104 . 105 ,

The second staffing term can be used to describe individual characteristics 203 for the state classifier (regardless of which data source 103 . 104 . 105 a feature 203 based). The second appointment term can be formulated as follows: J ₂ = C ₂ ∥W∥ _1,2 where C _{2 is} a hyperparameter by which a focus on the second population term, relative to other terms of the optimization problem, can be changed. The standard ∥W∥ _1,2 can be defined as ∥W∥ _1,2 = Σ _d ∥w _d ∥ ₂ where w _{d is} the d-th row of the weighting matrix W (and thus the d-th feature 203 ) corresponds. The second population term preferably causes all weighting values for a feature 203 are zero or all weighting values for a feature 203 are significantly different from zero. The second staffing term thus enables a clear selection of features 203 for a state classifier.

The o. G. Casting terms can also be referred to as Lasso (Least Absolute Shrinkage and Selection Operator) terms.

In sum, this results in the following optimization problem:

By solving this optimization problem, it is possible to determine a sparse weighting matrix W which unambiguously identifies the relevant and non-relevant characteristics 203 and / or data sources 103 . 104 . 105 result.

3 shows a flowchart of an exemplary method 300 for determining a state classifier. In step 301 become sensor data 111 from a variety of different sensors 103 . 104 . 105 (ie data sources) provided. These dates 111 can be preprocessed if necessary (step 302 ) and it can be a first part of the data 111 as training data for determining the state classifier and a second part of the data 111 be used as test data for the verification of the state classifier (step 303 ). From the training data can then a variety of training records {(x _i , y _i )} N / i = 1 be determined (step 304 ). For this purpose, based on the sensor data 111 a data source 103 . 104 . 105 Characteristics of one or more characteristics 203 be determined. Overall, this way for every data source 103 . 104 . 105 Characteristics for one or more characteristics 203 be determined. These features 203 are also referred to as originating features in this document.

Jtotal gelöst werden, um eine optimierte Gewichtungsmatrix W_opt zu bestimmen (Schritt 305). Auf Basis der optimierten Gewichtungsmatrix W_opt können dann die relevanten Merkmale 203 und/oder Datenquellen 103, 104, 105 bestimmt werden (Schritt 306). Zu diesem Zweck kann auf Basis der Gewichtungswerte w_d einer Zeile d ein repräsentativer Gewichtungswert w _d für das entsprechende Merkmal 203 ermittelt werden, (z. B. als Mittelwert, insbesondere als quadratisches Mittel, der einzelnen Gewichtungswerte w_dk, mit k = 1, ..., K, der Zeile d oder durch Auswahl von ein oder mehreren signifikanten Gewichtungswerten w_dk der Zeile d). Es können z. B. alle Merkmale d ausgewählt werden, für die der repräsentative Gewichtungswert w _d größer als ein vordefinierter Gewichtungs-Schwellenwert ist.From the training records {(x ₁ , y ₁ )} N / i = 1 As described above, the training data matrix X ^ and the state matrix Y can be determined. Furthermore, values for the hyperparameters C ₁ and C ₂ can be selected (step 305 ). It can then, for. B. by means of a gradient method, the above optimization problem

Jtotal to determine an optimized weighting matrix W _opt (step 305 ). On the basis of the optimized weighting matrix W _opt then the relevant features 203 and / or data sources 103 . 104 . 105 be determined (step 306 ). For this purpose, based on the weighting values w _{d of} a row d, a representative weighting value w _d for the corresponding feature 203 (eg as a mean, in particular as a quadratic mean, of the individual weighting values w _dk , k = 1, ..., K, row d, or by selecting one or more significant weighting values w _dk of row d ). It can z. For example, all features d for which the representative weighting value is selected w _d is greater than a predefined weighting threshold.

Possibly. Optionally, the determined weighting matrix W _opt can be multiplied on the right side by a centered training data matrix X 1 in order to transform the feature space into a subspace of the discriminant analysis (step 308 ).

By means of the weighting matrix W _opt , a subset of the D features 203 be selected for a state classifier. Furthermore, the state classifier can be based on the weighting values w _{dk of} the selected features 203 be determined. In particular, the weighting values w _dk can be used to weight a currently determined expression or a currently determined value of a (reduced) feature vector and to determine a weighted feature value (analogous to the matrix operation X 1, W). The weighted feature value can be determined for all possible classes k = 1,..., K (by means of the respective weighting values w _dk ). By comparing the weighted feature values, information is obtained about which class is most likely (analogous to the above class matrix Y). The most likely class (and corresponding driver state) can be selected.

Subsequently, a cross-validation of the determined state classifier can be carried out (step 310 ), where appropriate suitable hyperparameters are selected for this purpose (step 309 ). For the cross validation part of the training data can be used. Based on the cross-validation, a quality measure of the state classifier can be determined.

The above steps 305 . 306 . 307 . 308 . 309 . 310 can be repeated for different values of the hyperparameters C ₁ and C ₂ (eg for different values from a predetermined value interval). It is therefore possible to determine the values of the hyperparameters C ₁ and C ₂ for which the quality measure of the state classifier (ie for the classification quality) is maximized. It can therefore further optimize the selection of features 203 be carried out (step 311 ).

Finally, a validation of the optimal state classifier (which was determined on the basis of the optimal values of the hyperparameters C ₁ and C ₂ ) can be performed on the basis of the test data (steps 312 . 313 . 314 ). For this purpose, based on the test data test records {(x ~ _i , y _i )} T / i = 1 be determined (step 312 ). It corresponds x ~ _i an expression of the reduced feature vector (with a reduced number of features 203 ). T equals the number of test records. Possibly. Optionally, a transformation into the subspace of the discriminant analysis can take place (step 313 ). Furthermore, based on the test data sets, a quality measure for the determined state classifier can be determined (step 314 ).

• • ein Spektrum eines zeitlichen Ausschnittes 202 eines Elektrodensignals 111;
• • die Entropie eines zeitlichen Ausschnittes 202 eines Elektrodensignals 111; und/oder
• • eine fraktale Dimension eines zeitlichen Ausschnittes 202 eines Elektrodensignals 111.

The method described above 300 can be used in an analogous way to identify relevant channels of a data source 105 to select with a variety of sensor channels. In particular, an electroencephalograph includes 105 a plurality of electrodes or channels 405 (eg, 10-20, especially 14 or 19, electrodes) that are at different points on the skull 400 of a driver can be attached (see 4a ). The individual electrodes 405 may be considered as separate data sources (as defined in the procedure described in this document). In particular, from each electrode 405 (please refer 4a ) a separate sensor signal 111 recorded and provided. Every sensor signal 111 includes a certain current signal. Based on each sensor signal 111 can have one or more characteristics 203 be determined. Exemplary features 203 for the electrode signals 111 of an electroencephalograph are included

• • a spectrum of a temporal excerpt 202 an electrode signal 111 ;
• • the entropy of a temporal excerpt 202 an electrode signal 111 ; and or
• • a fractal dimension of a temporal section 202 an electrode signal 111 ,

For M electrodes 505 of an electroencephalograph (which may be considered M data sources or M channels, respectively) may each have Q characteristics 203 be determined, so that in total on the basis of the electrode signals 111 an electroencephalograph D = M · Q characteristics 203 can be determined.

By means of the method described above 300 can in one step the characteristics 203 and / or the electrodes or channels 405 be identified, which are relevant for a state classifier. Thus, the quality of an EEG data-based state classifier can be increased and / or the computational effort can be reduced.

4a . 4b and 4c illustrate the effect of common feature / data sources selection using the example of an electroencephalograph 105 , 4a illustrates the case where a pure feature-based selection occurs (this corresponds to the case where the first hyperparameter C ₁ = 0). This results in a relatively uniform distribution of the selected features 203 over a relatively large amount of selected electrodes or channels 415 of the electroencephalograph 105 (and a relatively high amount of computation involved). 4b illustrates the case where a pure data source-based, in particular electrode / channel-based, selection takes place (this corresponds to the case in which the second hyperparameter C ₂ = 0). In this case, relatively few electrodes / channels 415 selected, with all possible Q characteristics for each selected electrode 203 be taken into account. A data source-based selection typically results in a reduced computation effort, but may possibly be accompanied by a reduced quality of the determined state classifier.

4c shows by way of example the selection result of the described combined feature and data source selection (in which neither the first nor the second hyperparameter are zero, ie C ₁ ≠ 0 and C ₂ ≠ 0). Due to the combined feature and data source selection, an optimal compromise between computational effort and quality of the state classifier can be provided.

To train the state classifier, training data collected for a particular driver may be used. Thus, a state classifier with a high quality can be provided for a particular driver. Alternatively or additionally, the training data during the operation of a vehicle 100 from the control unit 101 be determined. The procedure described in this document 300 can then be based on the sensor data acquired during actual operation 111 be applied to adjust the state classifier. Thus, an adaptive state classifier can be provided which is addressed to the driver of the vehicle 100 adapts.

The procedure described in this document 300 allows sensor data in a computationally efficient manner 111 from a variety of data sources 103 . 104 . 105 . 405 to evaluate for a state determination. Thus, the condition of a driver can be reliably determined.

As explained above, in the context of the described method, a weighting matrix W _{opt is} determined which comprises different weighting values w _dk for different classes (ie for different states) of the driver. On the basis of these weighting values w _dk , it is possible to determine different states or state components of a driver at the same time. In particular, the different classes may describe different components of a driver's condition, such as: For example, a state component indicating a degree of driver's attention to the traffic and a state component indicating a degree of satisfaction of the driver. By providing corresponding classes and by determining the weighting matrix W _opt , information can be obtained for the individual classes at the same time for a plurality of state components on the basis of the weighting values w _dk .

The procedure described in this document 300 allows reliable and automatically noisy data sources 103 . 104 . 105 . 405 be identified and excluded for the determination of the driver's condition. This is effected in particular by the consideration of the first occupation term, by which data sources m 103 . 104 . 105 . 405 with low low weighting values W ^{m are} automatically excluded.

By the method described 300 In a single optimization step, a feature selection and a data source selection can be made. This simultaneous selection of features and data sources enables the determination of a global optimum.

Furthermore, it allows the described method 300 To reduce the number of actually consider features to really relevant features. Thus, without substantial loss of quality, the computational effort can be reduced to real-time applications in a vehicle 100 to enable. Furthermore, by reducing the number of data sources considered 103 . 104 . 105 . 405 the costs are reduced.

Furthermore, the selection of relevant channels / electrodes 405 in an electroencephalograph 105 an improved evaluation of brain activity for the determination of a driver's state allows.

The present invention is not limited to the embodiments shown. In particular, it should be noted that the description and figures are intended to illustrate only the principle of the proposed methods, apparatus and systems.

Claims (10)

Procedure ( 300 ) for determining a state classifier, the state classifier configured to assign a first state of a person to a first value of a reduced feature vector; the method ( 300 ), - providing ( 301 . 302 . 303 ) of training data, the sensor data ( 111 ) a plurality of different channels ( 405 ) of an electroencephalograph ( 105 ); - Determine ( 304 ), based on the training data, a plurality of training records; wherein a training record comprises a value of an origin feature vector and a corresponding actual condition of the person; wherein the source feature vector comprises a plurality of source features; wherein the plurality of source features at least one feature for each channel ( 405 ) of the plurality of different channels ( 405 ); - Determine ( 305 . 306 ), based on the plurality of training records, of weighting values for each originating feature; wherein a weighting value for an originating feature indicates a relevance of that originating feature to the state classifier; wherein the weighting values are determined by reducing or increasing an optimization function; wherein the optimization function comprises, - a first elimination term that rewards it if all the source characteristics of a particular channel ( 405 ) of the plurality of different channels ( 405 ) are irrelevant to the state classifier; and a second elimination term that rewards it when a single origin feature is irrelevant to the state classifier; - Choose ( 307 ), based on the weighting values, a subset of the plurality of source features for a reduced feature vector; and - providing ( 307 ) of the state classifier based on the weighting values for the selected subset of source features. Procedure ( 300 ) according to claim 1, wherein - the optimization function comprises an error term indicating an average deviation of predicted states of the person from actual states of the person; and - the error term determines a predicted state of the person for a training dataset by applying at least one weighting value to the value of the origin feature vector of that training dataset. Procedure ( 300 ) according to claim 2, wherein - the error term is determined based on the following formula: J = 1 / 2∥X ^ W - Y∥ 2 / F W is a weighting matrix comprising the weighting values; - X ^ is a training data matrix comprising the values of the source feature vector from the plurality of training records; Y is a state matrix indicating the actual states of the person from the plurality of training records; and - ∥ ∥ _F corresponds to a Frobenius norm. Procedure ( 300 ) according to any one of the preceding claims, wherein - determining ( 305 . 306 ) of weighting values, determining the first elimination term and determining the second elimination term; The weighting values comprise at least one weighting value for each originating feature of the plurality of originating features; Determining the first elimination term comprises determining a plurality of channel norms for the corresponding plurality of channels ( 405 ), where a channel standard for a channel ( 405 ) an L2 norm of the weighting values of the one or more source features of that channel ( 405 ); and determining the second elimination term comprises determining a plurality of feature norms for the corresponding plurality of origin features, wherein a feature standard for an origin feature is an L2 norm of the one or more weight values for that origin feature. Feature includes. Procedure ( 300 ) according to claim 4, wherein determining the first elimination term further comprises determining an L1 norm of the plurality of channel norms; and further comprising determining the second elimination term, determining an L1 norm of the plurality of feature norms. Procedure ( 300 ) according to one of the preceding claims, wherein - the first elimination term is determined on the basis of the following formula

W is a weighting matrix comprising the weighting values; M of the number of different channels ( 405 ) of the electroencephalograph ( 105 ) corresponds; Source _{m of} the set of original features of an mth channel ( 405 ) corresponds; and - w _q is a sub-matrix of the weighting matrix (only the weighting values of the one or more origin features for a particular m-th channel 405 ); and / or - the second elimination term is determined based on the following formula ∥W∥ _1,2 = Σ _d ∥w _d ∥ ₂ W _{d is} a sub-matrix of the weighting matrix that includes only the weighting values for a particular origin feature. Procedure ( 300 ) according to one of the preceding claims, wherein - the first elimination term and the second elimination term are taken into account via respective factors C ₁ and C ₂ in the optimization function; and - the method ( 300 ), determining ( 305 . 309 . 310 . 311 ) of values for the factors C ₁ and C ₂ , so that a quality of the determined state classifier is increased. Procedure ( 300 ) according to one of the preceding claims, wherein - the electroencephalograph ( 105 ) 10 or more channels ( 405 ); and / or the sensor data ( 111 ) of a channel ( 405 Show brainwaves of the person. Procedure ( 300 ) according to one of the preceding claims, wherein The weighting values comprise at least one weighting value for each origin characteristic of the plurality of originating features; and / or - the weighting values comprise at least one weighting value for each possible state of a plurality of predefined states. Procedure ( 300 ) according to one of the preceding claims, wherein the plurality of source features - multiple source features for each channel ( 405 ) of the plurality of channels ( 405 ); - corresponding source characteristics for the different channels ( 405 ); and / or - one or more of the following characteristics of origin: - a spectrum of a temporal excerpt ( 202 ) of the sensor data ( 111 ) of a channel ( 405 ); - an entropy of a temporal excerpt ( 202 ) of the sensor data ( 111 ) of a channel ( 405 ); and / or - a fractal dimension of a temporal section ( 202 ) of the sensor data ( 111 ) of a channel ( 405 ).

Images