DE102022203803B4 - Method and monitoring system for monitoring a manual manufacturing process and method for creating a machine learning facility - Google Patents

Abstract

Method for monitoring an at least partially manual manufacturing process, wherein- while a person (10) is performing a manual activity as part of the manufacturing process, brain signals of the person (10) are measured by means of a portable brain signal detector (1),- the measured brain signals (12) are automatically processed by means of a machine learning device (9) and/or a time series analysis that has been trained for this purpose, and- the performance of the manual activity is evaluated with regard to a predefined, intended performance as or based on an output (13) of the machine learning device (9) and/or the time series analysis,- several phases of the manual activity are recognized on the basis of the brain signals (12), and as part of the evaluation (13, 14), it is determined in which phase a deviation from the intended performance occurs.

Description

The present invention relates to a method and a monitoring system for monitoring a manual manufacturing process. The invention further relates to a corresponding training method for a machine learning device.

In the partially automated, continuous mass production of parts and assemblies, individual production steps are carried out manually by workers or workers, i.e. people. This can be, for example, handling, i.e. inserting, parts or components, checking, i.e. checking for the presence of certain parts or features and/or other process-supporting activities. Naturally, errors can occur, for example due to inattention, tiredness and the like. There are already various error prevention methods designed to ensure that manual activities are carried out without errors. For example, electrical information signals or mechanical constraints (Poka Yoke) can be used for this purpose. Despite these measures, however, errors can still occur by people, i.e. human workers, which can lead to faulty production or damage to a production facility due to incorrect operation and the like. This can lead to additional work, delays and/or increased component rejects, particularly in time-critical production processes. Such errors can occur primarily through carelessness due to a routine of repetitive work or activities that develops over time.

In order to address the problems mentioned above, or at least to detect errors, there is a need for improved process monitoring. One approach to this is described by DE 10 2019 104 822 A1 a method and a device for monitoring an industrial process step. In this, digital image data is recorded using a corresponding monitoring system. A current process state is also determined by generating the current process state as output data from the recorded digital image data as input data based on a correspondingly trained decision algorithm. The process step is then monitored by generating a visual, acoustic and/or haptic output depending on the determined current process state. This is intended to achieve improved monitoring of manual process steps in an industrial process with regard to quality assurance.

A method for monitoring a manufacturing process is also available in the DE 10 2017 207 992 A1 described. In it, a workpiece is detected with a sensor unit and information detected by the sensor unit is sent to a database, where a workpiece-specific data set is created. The workpiece is detected by the sensor unit before, during and/or after a processing step or a handling step on the workpiece is carried out. The data set stored in the database is then updated before, during and/or after a processing step or handling step on the workpiece is carried out. The aim is to provide a method with which the disadvantages that occur during the individual production of workpieces can be reduced and/or process reliability can be increased.

A concrete industrial application for monitoring is, for example, in the DE 10 2016 120 131 A1 described. A method for supporting a sorting process of workpieces arranged on a sorting table, which were produced with a flatbed machine tool using a processing plan, is disclosed therein. In this, a processing image data set of the processing plan on which the arrangement of at least one workpiece was based is first provided. The sorting table with a plurality of workpieces arranged next to one another is then imaged and a first sorting image data set is generated. The sorting table is then repeatedly imaged and a second sorting image data set is generated after at least one workpiece has been removed from the sorting table. The first sorting image data set and the second sorting image data set are then compared using the processing image data set, a sorting signal being generated that contains information that includes the type, position and/or shape of the removed workpiece. This is intended to facilitate assignment, reduce errors in assignment and/or enable subsequent processes to be carried out efficiently.

Further technical background is provided by US 2021 / 0 248 514 A1. It describes a computer-implemented method for selecting and configuring an automated robotic process. It identifies a type of thought process when performing a task based on a temporary biometric measurement of a worker.

Next, a component of a Kl solution is selected to replicate this type of reasoning. This selection is based on an identified active area of the worker's neocortex.

Further technical background is the US 11,157,081 B1 in which a wearable device is described. The device therein comprises a display for providing information to a respective wearer, wherein the information comprises at least two selectable options. The device further comprises a brain monitoring device for tracking a brain wave activity of the wearer and for detecting a mental decision. Based thereon, one of the options is to be identified as a selected option. The device further comprises an electromyograph which is configured to track the options based on the brain wave activity in combination with muscle activity tracked by the electromyograph.

The object of the present invention is to enable a particularly efficient and cost-effective production of goods with human involvement.

This object is achieved according to the invention by the subject matter of the independent patent claims. Possible embodiments and further developments of the present invention are disclosed in the dependent patent claims, in the description and in the figures.

The method according to the invention can be used to monitor an at least partially manual manufacturing process. The method can therefore be used, for example, for error detection or process control in the context of a manufacturing process or a process that includes one or more manual manufacturing, process or work steps. In particular, the method can be used in industrial manufacturing, but is not limited to this area of application. The method can therefore be used in various areas of application, in particular in applications with repetitive, i.e. frequently repeated, similar manual activities, processes or actions. In one step of the method according to the invention, while a manual activity is being carried out by a person in the context of the manufacturing process, the person's brain signals are measured using a portable brain signal detector. These brain signals can be, for example, brain waves, brain currents, voltage fluctuations on the outside of the head and/or other electromagnetic signals related to the person's brain activity. The measured brain signals are therefore measurement data that indicate or describe the activity of the brain of the respective person. Since the activity, i.e. actions, movements and/or the like of the person are controlled by their brain, they are encoded or represented in the brain signals. For example, EEG electrodes, probes, sensors, micro or nano wires and/or the like can be used to measure the brain signals. These can be integrated into the portable brain signal detector, which can be designed to be worn or attached to a person's head. The portable brain signal detector can be designed, for example, as a cap, a net that can be placed on the head, a helmet, a headset, a head-mounted display (HMD) or the like.

In a further method step of the method according to the invention, the measured, i.e. recorded, brain signals are automatically processed by means of a machine learning device trained for this purpose and/or by means of a time series analysis. Such a machine learning device can be or comprise, for example, an artificial neural network, a random forest classifier and/or the like. The machine learning device can therefore be designed as a software component or comprise a software component. The machine learning device can also be designed as a hardware device, for example with a data memory in which a corresponding software component is stored, or as a hardware circuit. The measured brain signals are thus - possibly after signal preprocessing - fed or provided to the machine learning device as input, i.e. as input data. The measured brain signals - possibly after signal preprocessing - can also be fed or provided as input to a device that is set up to carry out a predetermined time series analysis, in particular automatically or semi-automatically. Such a time series analysis in the present sense can in particular apply a given classical algorithm, i.e. one not based on machine learning. For example, the time series analysis can be based on the DTW algorithm and/or include a determination and predetermined evaluation of distance measures and/or the like.

In a further method step of the method according to the invention, the performance of the manual activity is evaluated as an output, i.e. initial data of the machine learning device or based on such an output, with respect to, i.e. with regard to or relative to a predefined intended, i.e. intended or correct, performance of the manual activity. This can be done automatically or partially automatically. For example, the device The machine learning device and/or the time series analysis can output this directly as an output or indicate whether the activity was carried out as intended or in a manner deviating from its intended performance, for example incorrectly or irregularly. The machine learning device can therefore be trained and/or the time series analysis device can be set up to recognise corresponding deviations or errors in the performance of the manual activity by the respective person based on their brain signals. This is possible because an action that represents an error in the performance of the manual activity is typically accompanied by a brain signal that does not occur or occurs in a different form when the manual activity is carried out as intended without this action or error.

In the present invention, several phases of the manual activity are recognized on the basis of the measured brain signals, for example automatically by the machine learning device and/or the time series analysis. Such phases can, for example, correspond to different sub-steps, movements or hand movements, the chaining of which can result in the manual activity or a cycle of the manual activity. Furthermore, as part of the assessment, it is then determined, in particular automatically, in which phase a deviation from the intended performance of the respective manual activity occurs, has occurred or is likely to occur. For this purpose, separate training data can be generated for the individual phases, for example, and used to train the machine learning device. Likewise, training data can be generated or used to train the machine learning device that includes or depicts several phases or at least one complete cycle of the manual activity, wherein areas corresponding to different phases and/or points or areas between successive phases can be marked or defined therein. The training data can therefore be annotated accordingly. The machine learning facility can also be trained to recognize or differentiate between different phases of the respective manual activity. The design of the present invention proposed here enables deviations or errors or types of errors to be detected particularly precisely and specifically. This can enable correspondingly precisely adapted reactions, safety or corrective measures and thus reduce the corresponding effort and/or help to avoid waste.

The output of the machine learning device and/or the device for time series analysis can be further processed or post-processed, for example, using another predetermined, in particular classic, i.e. not machine learning-based, algorithm or method. For example, filters, in particular a low-pass filter, can be applied, a pattern comparison, signal processing, a threshold comparison, for example for a peak height in the brain signals or a degree of deviation or similarity, for example between the measured brain signals and a predetermined brain signal or brain signal pattern recorded when the manual activity was carried out as intended, for a confidence value regarding the intended or incorrect or irregular performance of the manual activity and/or the like can be carried out in order to carry out or obtain the assessment. A filter can be used to dampen or filter out prediction fluctuations of the machine learning device if necessary. Further or post-processing can therefore enable a more robust assessment, for example.

The measured brain signals can therefore be processed using a classic or machine learning-based method or device. Processing can also be carried out using a classic method or a classic data processing device and using a machine learning method or device. In the latter case, respective outputs, i.e. processing or intermediate results, can be fused. Known data or sensor fusion methods can be used for this purpose. By merging the output of two different methods or devices in this way, an improved robustness, quality or reliability of the assessment can be achieved.

The evaluation can, for example, be output or provided and/or a predefined control signal or a control signal automatically selected from several predefined control signals depending on the evaluation can be generated, output or triggered automatically. For example, when it is detected that the manual activity is not being carried out as intended, a corresponding information or warning signal can be automatically output or triggered.

The present invention enables overall implementation of error prevention and support measures men through real-time analysis of the brain signals of the respective person while performing the manual activity. The present invention can therefore open up a new field of monitoring and process control of processes in which people are integrated or in which people are involved, compared to previous approaches.

Research results have shown that, especially in the case of repetitive work or activities, brain waves can show the same, repeating signal pattern. If there is a deviation from a corresponding standardized activity, this signal pattern will change. By detecting this change in the brain signals, a corresponding deviation, i.e. potentially an error when performing the manual activity, can be recognized or predicted. This means that the person in question can be warned, for example, of an impending or looming error. This not only enables errors to be recognized, but also proactively avoided. The latter is the case because a corresponding change in the brain signals can occur and be detected before or at the start of a movement or action that deviates from the intended performance and can at least potentially lead to an error. The present invention can thus extend previous error prevention measures to the cognitive evaluation of brain signals of the person performing the manual activity. The present invention can thus form a further level of security, in particular for time-critical, error-prone or repetitive activities, for example in the continuous, industrial series production of components.

The present invention can also enable cost savings for error prevention measures by reducing other, conventional measures and by reducing corresponding measurement technology for conventional product and process monitoring. In addition, the present invention can enable a reduction in rejects and production delays by improving the integration of people into production lines or a respective manufacturing process. This can also potentially result in shorter production times by eliminating other control measures.

Previous error prevention measures include Poke Yoke, comparatively expensive and complex measuring technology for checking product and process properties, relatively expensive and complex lighting technology for visual support and/or the like. Such solutions often have the disadvantage that they involve a relatively large amount of design effort and relatively high costs. In addition, in many cases this can extend the process times in production plants and circumvention of the measures can often not be completely avoided or ruled out. This has led to the approach of continually increasing the level of automation of plants in order to avoid errors that are induced or caused by people, thus making the role of people in production increasingly insignificant. In contrast, the present invention can also help reduce errors when people are involved in production processes and thus maintain or increase the importance of people in production. This can have positive effects, for example counteracting long-term job cuts in production.

The method according to the invention can be used flexibly in different situations and scenarios. For example, the processing of the measured brain signals and/or the corresponding evaluation can be carried out continuously or for time intervals of a predetermined length. The length of the time intervals can be adapted to the respective manual activity, for example, i.e. correspond to the length of an activity step or a repeating cycle of the manual activity or a corresponding predetermined or learned duration or average duration. The adaptation of the length of the time intervals can enable the method to be flexibly adapted to different applications, for example depending on the respective complexity of the manual activity. Likewise, for example, a prediction horizon for the detection of impending or - at least presumably - imminent errors or deviations from the intended performance of the respective manual activity can be adapted accordingly by the machine learning device and/or the time series analysis device. For example, a shorter forecast horizon can enable more reliable fault detection or fault prediction, while a longer forecast horizon can enable particularly early reactions or particularly early initiation of safety or corrective measures.

In one possible embodiment of the present invention, several different signal streams are measured simultaneously as the brain signals. Accordingly, these several simultaneously measured signal streams can be used as input for the machine learning and/or time series analysis device, i.e. fed to or provided for this purpose. The several signal streams can be, for example, These can be signal or data streams from several identical or different sensors, from several measuring points or measuring locations, for different signal types, frequency ranges, voltage levels and/or the like. Measuring and using several signal streams can enable a more precise recording or mapping of the manual activity and thus a more precise and reliable assessment of the performance of the respective manual activity or a corresponding error detection.

In a further possible embodiment of the present invention, the brain signals are measured using a head-mounted display (HMD) that has at least one sensor for measuring the brain signals. In particular, the HMD can be designed as AR glasses (AR: Augmented Reality). A corresponding sensor for measuring the brain signals can, for example, be integrated into or arranged on a temple or head strap or the like of the HMD. In comparison to other solutions, an HMD can be particularly easy to handle and also opens up the possibility of measuring brain signals not only at one measuring point, but over a spatially extended area along the head of the respective person in a particularly simple manner. This can enable a correspondingly improved accuracy and reliability or a correspondingly improved level of detail of the measured brain signals and thus ultimately also a correspondingly improved, i.e. more precise and/or more reliable, assessment. The HMD also inherently offers the possibility of providing the respective person, i.e. the respective wearer of the HMD, with individual feedback regarding the brain signals and/or the performance of the manual activity or its evaluation, i.e. for example displaying them, in particular at least essentially independently of the respective environmental conditions and while preserving privacy or data protection requirements, even if there are other people in the vicinity of the respective person.

In a further possible embodiment of the present invention, a warning is automatically issued to the respective person performing the manual activity if the assessment indicates a deviation from the intended performance of the manual activity. For example, a degree and/or a type of deviation can be taken into account. For example, a corresponding warning can then only be issued if the deviation is greater than a predetermined deviation threshold and/or if the deviation is of a predetermined type or the like. In this way, correspondingly smaller individual, i.e. person-dependent, characteristics or deviations in the performance of the respective manual activity can be tolerated, which can, for example, depend on the day's form and usually do not lead to an error or problem in the production process. This ultimately avoids unnecessary warnings and enables the respective person to perform the manual activity individually or slightly varied and thus easier or more pleasantly, without distracting or irritating the respective person with ultimately unnecessary warnings. On the other hand, the automatic output of the warning enables an immediate reaction to an error that has occurred or is about to occur at the location where it occurs or is caused. This makes it possible to react quickly and locally to detected deviations from the intended performance of the respective manual activity or resulting errors in production. This can enable a particularly effective and efficient avoidance or correction of errors when performing the respective manual activity or in production, which can ultimately also help to avoid rejects or production errors.

It can also be provided that if the assessment shows or indicates that the manual activity is being carried out as intended, i.e. correctly, a corresponding positive note indicating this is issued to the person concerned. This can support training or familiarization of people in the respective manual activity through positive feedback and/or improve motivation. This in turn can also contribute to a reduced error rate.

In a possible development of the present invention, the warning and/or the positive indication is issued using the same device by means of which the brain signals are measured. For example, both the brain signals can be measured and the warning issued using the HMD mentioned elsewhere or the like. The development of the present invention proposed here can enable a particularly flexible application of the present invention that is at least essentially independent of external devices and/or environmental conditions. For example, the warning and/or the positive indication can be perceived particularly reliably by the respective person even when there is increased ambient noise in the environment and/or an external warning or indication device cannot be easily arranged in the person's typical field of vision when carrying out the activity. net. In addition, this makes it particularly easy and reliable to issue the warning and/or positive information to a specific person. This can prevent distraction or confusion for other people in the area and enables a particularly high level of data protection.

Irrespective of the other described embodiments of the present invention, the actual evaluation or processing of the measured brain signals can, depending on the embodiment or the technical possibilities available in the individual case, be carried out by the same device, for example also by the HMD, which is also used to measure the brain signals and/or to output the warning mentioned elsewhere, or can be outsourced to an external data processing device, for example a central computer or server or the like. The latter can reduce the energy requirements of the portable device, so that it can be designed to be lighter and/or less complex and thus, for example, more comfortable to wear and/or more cost-effective. In addition, the external evaluation or processing of the measured brain signals can enable or simplify the use of more computing resources for this, which in turn can enable a correspondingly more accurate and reliable and/or faster evaluation or error detection.

A further aspect of the present invention is a method for creating or training a machine learning device that is set up, i.e. in particular trained, for automatic or at least partially automated detection of errors when a person performs a manual activity. Reference brain signals are measured as training data during several error-free performances of the manual activity. These can be brain signals from the same person who will also perform the manual activity in the final application or productive use, brain signals from another person, or brain signals from several people. The former can enable individually coordinated detection of deviations and thus a correspondingly accurate assessment, while the latter can take differences between different people into account and thus enable particularly robust deviation detection or assessment, for example if different people are to or will perform the manual activity without each adjustment of the machine learning device. The machine learning device is trained using this training data comprising the reference brain signals. Here, the machine learning facility can be trained to recognize the intended performance of the manual activity based on the reference brain signals. Conventional training methods can be used for this, such as a backpropagation procedure and/or the like.

The machine learning device created or trained by means of the method according to the invention for creating a machine learning device can in particular be or correspond to the method according to the invention for monitoring an at least partially manual manufacturing process.

In the present invention, error reference brain signals are also measured as training data during an error-prone manual activity, i.e. one that deviates from the intended exercise. As explained in connection with the reference brain signals, these error reference brain signals can be or include brain signals from the same person who will also carry out the manual activity in the final application or productive use, brain signals from another person and/or brain signals from several people. The error reference brain signals are then also used to train the machine learning device. In a possible development of the present invention, the error reference signals can be measured for several different errors. In this case, only exactly one single error or one single type of error can be made, i.e. measured, per individual exercise or per cycle of the manual activity. This can not only enable the detection of any deviation, but also its identification. This can, for example, enable error- or deviation-specific warnings, safety or corrective measures or the like. In the present invention, the machine learning device can be trained to detect an error-prone or improper performance of the manual activity, in particular to detect several different errors or types of errors. This can enable a more precise and reliable detection of deviations and thus ultimately a correspondingly improved evaluation. In order to simplify and accelerate the recording of the error reference brain signals, the respective person on whom the error reference brain signals are measured can, for example, be instructed to make a certain predetermined error, i.e. to incorporate it into the performance of the manual activity. The error reference brain signals can thus be This means that the training data can be annotated in a simple and low-effort manner, i.e. a simple or automatic association between a specific error or a specific type of error or deviation from the intended performance and the associated error reference brain signals can be achieved. This can enable or support a particularly simple, fast and low-effort generation of training data.

A further aspect of the present invention is a monitoring system for monitoring an at least partially manual manufacturing process. The monitoring system according to the invention comprises a portable brain signal detector, which can be or comprise, for example, the device mentioned elsewhere, i.e., for example, the HMD. The monitoring system according to the invention further comprises a data processing device for automatically processing brain signals measured by means of the portable brain signal detector. According to the invention, it is provided that this data processing device is or comprises a suitably trained machine learning device created according to the method according to the invention and/or a device set up for a, in particular automatic or semi-automatic, predetermined time series analysis of brain signals and/or that the monitoring system according to the invention is set up for, in particular automatic or semi-automatic, execution of the method according to the invention for monitoring an at least partially manual manufacturing process. For this purpose, the monitoring system, in particular its data processing device, can comprise, for example, a processing device, i.e., for example, a microchip, microprocessor or microcontroller or the like, and a computer-readable data storage device coupled thereto. An operating or computer program can then be stored in this data storage device, which encodes or implements the method steps, measures or sequences or corresponding control instructions described in connection with the method according to the invention for monitoring the at least partially manual production process. This operating or computer program can then be executable by the process device in order to carry out the corresponding method or to effect its execution. The data processing device can be part of the portable brain signal detector, i.e. integrated into it. The data processing device can also be a data processing device external to the portable brain signal detector. In the latter case, the portable brain signal detector can send data, in particular the measured brain signals, to the data processing device, for example via a corresponding data cable or a wireless data connection, for example a radio, WLAN or Bluetooth data connection or the like. The data processing device can also be set up to send the resulting evaluation or a corresponding indication or the like to the portable brain signal detector and/or an output device.

In a possible development of the present invention, the portable brain signal detector is designed as AR glasses with at least one integrated sensor for measuring electrical brain signals of a respective wearer. This can enable the implementation of a feedback loop and/or the transmission of warnings or the like to the respective wearer in a particularly simple and reliable or personalized manner due to the AR glasses' ability to output information to the respective wearer.

Further features of the invention can be seen from the following description of the figures and from the drawing. The features and combinations of features mentioned above in the description as well as the features and combinations of features shown below in the description of the figures and/or in the figures alone can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the invention.

• 1 eine schematische Darstellung zur Veranschaulichung eines Überwachungssystems für manuelle Fertigungsschritte; und
• 2 eine schematisch Übersichtsdarstellung zur Veranschaulichung eines Verfahrens zum Überwachen eines zumindest teilweise manuellen Fertigungsprozesses mittels eines entsprechenden Überwachungssystems.

The drawing shows in:

• 1 a schematic diagram illustrating a monitoring system for manual production steps; and
• 2 a schematic overview to illustrate a method for monitoring an at least partially manual manufacturing process by means of a corresponding monitoring system.

Identical or functionally equivalent elements are provided with the same reference symbols in the figures.

1 shows a partial schematic overview of a monitoring system for monitoring an at least partially manual production process. The monitoring system here comprises a portable brain signal detector 1, which is designed here as an example as AR glasses. The brain signal detector 1 has an output device 2 for displaying or outputting visual and/or acoustic information. Furthermore, on the headband or glasses bracket 3 of the portable brain signal detector 1, here the AR glasses, sensors 4 for measuring signals that represent a brain activity of the respective wearer are arranged. The brain signal detector 1 can also have data processing means that are not shown in detail here, for example a microprocessor or the like for processing the measured signals and/or for sending the measured signals or processed or preprocessed signals generated therefrom.

Additionally or alternatively, the measured signals can be processed outside the portable brain signal detector 1. For this purpose, a corresponding external data processing device 5 is shown schematically here. This can receive the measured signals from the portable brain signal detector 1 via an interface 6 and process them using a processor 7 and a data storage device 8. For this purpose, a machine learning component 9, for example an appropriately trained artificial neural network or the like, can be stored in the data storage device 8, as indicated here by way of example. The data processing device 5 can also be set up to carry out a time series analysis of the signals measured by means of the brain signal detector 1. The measured signals can thus be processed in order to recognize deviations from a specified, predefined or predetermined intended performance of the manual activity that occur or are imminent when a wearer of the portable brain signal detector 1 performs a manual activity, i.e. corresponding errors or signs of errors, or to evaluate the measured signals or the corresponding performance of the manual activity accordingly.

In the present case, it is therefore intended to use artificial intelligence, i.e. in particular a machine learning device such as the machine learning component 9, to detect deviations in a manual activity from its intended performance and, if necessary, to prevent these deviations in advance, based on brain waves, brain wave patterns or the like of a wearer of the brain signal detector 1, for example a fitter or worker. 2 a schematic overview for further illustration. Here, a person 10 is indicated who is wearing the portable brain signal detector 1. This provides an input 11, which here comprises several brain signals 12 measured simultaneously. The brain signal detector 1, in this case the AR glasses, is therefore equipped with additional sensors compared to conventional AR glasses, namely the sensors 4, which can be attached to the frame of the AR glasses, for example. The AR glasses are used here to record the brain waves or brain wave patterns, i.e. the brain signals 12, via the frame area. The resulting multivariate signal pattern of brain signals 12 is then processed by means of the machine learning component 9, whereby the machine learning component 9 is directly in the portable brain signal detector 1 or alternatively as in 1 indicated can be implemented or stored in an external data processing device 5.

The processing of the brain signals 12 by means of the machine learning component 9 can result in an assessment or evaluation 13 for a state of the respective manual activity, for example a corresponding manufacturing or production step. This assessment or evaluation 13 can be determined continuously or in a sequence for a specific time interval, i.e. for a section of the brain signals 12.

The length of these time intervals or sections and, if applicable, a prediction horizon of the assessment or evaluation 13 can be selected depending on the respective application case or determine a complexity of the model used, in particular the machine learning component 9.

The evaluation 13 and/or a hint 14 derived therefrom can be output to the person 10 via the output device 2. The hint 14 can, for example, point out or draw the person 10's attention to a detected or predicted error, a corresponding safety or corrective measure or, for example as part of training the person 10 in the respective manual activity, to possible improvements or the like.

Before such productive use, the machine learning component 9 can first be trained. To do this, the person 10 and/or at least one other person can carry out the respective manual activity without errors. In the meantime, reference brain signals corresponding to this error-free execution of the manual activity can be recorded by means of the brain signal detector 1 and training data can be generated or provided for the machine learning component 9 based on this. The person 10 can thereby carry out the respective manual activity preferably with maximum concentration and without errors several times in succession. This execution of the respective manual activity can be supported by the AR glasses, in particular their output device 2, for example by displaying corresponding assistance, hand and/or component positions, movement paths, control element markings, target positions and/or or the like. Furthermore, error reference brain signals can be recorded and converted into corresponding training data. These error reference brain signals can be recorded by means of the brain signal detector 1 while the person 10, for example intentionally or accidentally or inadvertently, makes errors when carrying out the manual activity. Brain signals 12 corresponding to such errors or recorded during such errors can be or have been marked accordingly in the training data.

The training data can be used to train the machine learning component 9, for example by means of a supervised learning method, to recognize brain signals corresponding to the intended performance of the respective manual activity and brain signals deviating therefrom and, if necessary, to automatically output corresponding feedback to the brain signal detector 1 or the output device 2 or the person 10 in real time.

If necessary, the functionality of the monitoring system can then be checked in a test phase and, if necessary, further training data can be generated and/or the training of the machine learning component 9 can be continued or refined.

The monitoring system with the machine learning component 9 trained in this way can then be used, for example, in industrial manufacturing or production and support the respective person 10, for example a worker or worker, in manual activities or manufacturing or process or production steps. The use of AR glasses is then not necessary, so that other types of sensor devices, communication means, interfaces or the like can also be used if necessary.

Overall, the examples described show how AI-based quality optimization of partially automated production processes can be realized.

list of reference symbols

1

brain signal detector

2

output device

3

temples

4

sensor

5

data processing facility

6

interface

7

processor

8

data storage

9

machine learning component

10

person

11

input

12

brain signals

13

Evaluation

14

Notice

Claims (9)

Method for monitoring an at least partially manual manufacturing process, wherein - while a person (10) is performing a manual activity as part of the manufacturing process, brain signals of the person (10) are measured by means of a portable brain signal detector (1), - the measured brain signals (12) are automatically processed by means of a machine learning device (9) and/or a time series analysis that has been trained for this purpose, and - the performance of the manual activity is evaluated with respect to a predefined, intended performance as or based on an output (13) of the machine learning device (9) and/or the time series analysis, - several phases of the manual activity are recognized on the basis of the brain signals (12), and as part of the evaluation (13, 14), it is determined in which phase a deviation from the intended performance occurs. procedure according to claim 1 , characterized in that several different signal streams (12) are measured simultaneously as the brain signals (12). Method according to one of the preceding claims, characterized in that the brain signals (12) are measured by means of a head-mounted display which has at least one sensor (4) for measuring the brain signals (12), in particular by means of AR glasses (1). Method according to one of the preceding claims, characterized in that a warning (14) is automatically issued to the respective person (10) performing the manual activity if the evaluation (13) indicates a deviation from the intended performance of the manual activity, and/or a positive indication (14) is issued if the evaluation (13) indicates that the manual activity is being performed as intended. procedure according to claim 4 , characterized in that the warning indication (14) and/or the positive indication (14) is issued by means of the same device (1) by means of which the brain signals (12) are measured. Method for creating a machine learning device (9) which is designed to detect errors when a person (10) is performing a manual activity, wherein reference brain signals (12) are measured as training data during several error-free performances of the manual activity and the machine learning device (9) is trained therewith, wherein error reference brain signals (12) are also measured as training data during an error-prone performance of the manual activity and the machine learning device (9) is also trained therewith. procedure according to claim 6 , characterized in that the error reference brain signals (12) are measured for several different errors during the error-prone performance of the manual activity and the machine learning device (9) is also trained therewith. Monitoring system (1, 5) for monitoring an at least partially manual manufacturing process, comprising a portable brain signal detector (1) and a data processing device (5) for automatically processing measured brain signals (12), wherein the data processing device (5) has a correspondingly trained, according to a method according to claim 6 or 7 created machine learning device (9) and/or a device which is designed to carry out a predetermined time series analysis of measured brain signals, and/or wherein the monitoring system (1, 5) is designed to carry out a method according to one of the Claims 1 until 5 is set up. Monitoring system (1, 5) according to claim 8 , characterized in that the portable brain signal detector (1) is designed as AR glasses (1) with at least one integrated sensor (4) for measuring electrical brain signals (12) of a respective wearer (10).

Images