KR20240110157A - Digital twin based-site management server and site management method for monitoring safefy acctident in industrial site - Google Patents

Abstract

A digital twin-based industrial site monitoring system and site management method for safety accident management is launched. The field management server according to the present invention is included in an industrial site monitoring system, and includes 2D images from a plurality of cameras deployed at the industrial site and log data from one or more mobile robots deployed at the industrial site - the log data is time-sensitive. Includes the location, path, direction and collision level of the mobile robot according to the flow of - a data communication unit that collects, a data recording unit that cumulatively records the 2D image and the log data, and can operate on the data communication unit and the data recording unit. It includes a processor that is tightly coupled. The processor detects the occurrence of an accident event in the industrial site based on the 2D image and the log data, and stores accident description information indicating the time and location at which the accident event occurred as an accident history in the data recorder. It is configured to be stored in .

Description

Digital twin-based site management server and site management method for safety accident monitoring at industrial sites {DIGITAL TWIN BASED-SITE MANAGEMENT SERVER AND SITE MANAGEMENT METHOD FOR MONITORING SAFEFY ACCTIDENT IN INDUSTRIAL SITE}

The present invention relates to a digital twin-based field management server and field management method. More specifically, the present invention relates to a field management server and field management method based on digital twins, and more specifically, to analyze field data collected from multiple cameras and one or more mobile robots placed within an industrial site to prevent safety accidents at industrial sites. Immediately determine whether or not the accident occurs, minimize blind spots that exist in industrial sites through mutual complementation between camera images and robot data, identify the cause of accidents, and automatically implement safety measures (measures) to prevent future safety accidents. It is about establishing and implementing techniques.

As the Serious Accident Punishment Act comes into effect from January 2022 with the purpose of reducing various safety accidents at industrial sites and raising awareness of those involved, safety accidents at industrial sites will be prevented in advance and, if a safety accident unavoidably occurs, the relevant accident It has become more important to accurately and quickly identify the cause.

The industrial facility management system according to Patent Document 1 remotely monitors and controls industrial facilities across the country through complex technology and supports optimized operation of industrial facilities and resources through database construction. In addition, according to Patent Document 2, a process management system using middleware capable of tracking process data can save time and cost in building a system, and can acquire data from all devices used in the factory, helping to plan production. Helpful. Additionally, according to Patent Document 3, methods and systems for detection in an industrial Internet of Things data collection environment with large data sets utilize collected data for monitoring, remote control, autonomous action, and other activities in the industrial environment. do.

Figure 1 is a schematic diagram of an industrial site according to the prior art. Inside an industrial site where a large number of facilities are deployed, a mobile robot (AGV) and a worker (U), each with a unique cognitive range, touch equipment or equipment while performing their respective roles. As the cognitive range is limited due to other structures, etc., collision accidents may occur as a result of failure to detect and avoid the opponent.

However, in industrial sites where conventional technology is applied, it is only possible to check images acquired through filming in a very limited space, so it is not possible to determine immediately or after the fact whether various safety accidents that may occur due to carelessness, poor workmanship, or system errors have actually occurred. However, there are some aspects that are insufficient to detect it. Moreover, once a safety accident has occurred, it is not easy to determine when, why, and how seriously the safety accident occurred due to blind spots formed throughout the industrial site, as well as to prevent recurrence of similar safety accidents in the future. It does not provide any kind of guide.

Patent Document 1: Registered Patent Publication No. 10-0448668 (2004.09.03) Patent Document 2: Registered Patent Publication No. 10-1724557 (2017.04.03) Patent Document 3: Public Patent Publication No. 10-2020-0037816 (2020.04.09)

The present invention inputs 2D images taken at various angles, records log data about the field device of the mobile robot controlled by the terminal, renders and converts the 2D images into 3D images, synchronized device status, and neural rendering image model. The purpose is to provide an industrial site monitoring system that predicts and prevents collision accidents between mobile robots and workers.

Another purpose of the present invention is to provide an industrial site monitoring system that predicts the occurrence of accidents in 3D images and provides guidance to mobile robots and workers to prevent recurrence of accidents.

Other objects and advantages of the present invention can be understood from the following description, and will be more clearly understood by practicing the present invention. In addition, it will be readily apparent that the objects and advantages of the present invention can be realized by means and combinations thereof indicated in the claims.

The field management server according to one aspect of the present invention is included in an industrial site monitoring system, and includes 2D images from a plurality of cameras deployed at an industrial site and log data from one or more mobile robots deployed at the industrial site - the log Data includes the location, path, direction, and collision level of the mobile robot over time - a data communication unit that collects, a data recording unit that cumulatively records the 2D image and the log data, and the data communication unit and the data recording unit. It includes a processor operably coupled to. The processor detects the occurrence of an accident event in the industrial site based on the 2D image and the log data, and stores accident description information indicating the time and location at which the accident event occurred as an accident history in the data recorder. It is configured to be stored in .

The processor, when the occurrence of the accident event is detected, renders the 2D image into a 3D image based on the accident description information of the accident event to generate an accident reproduction image for at least a portion of the industrial site. You can.

The processor may input accident description information of the accident event into a cause analysis model. The cause analysis model may be trained in advance to classify the accident explanation information into one of a plurality of clusters mapped to different accident causes. The processor may be configured to map an accident classification code previously assigned to an accident cause mapped to a cluster classified by the cause analysis model with respect to the accident description information to the accident event.

The processor may update accident guide information of a sub-area where the accident event occurred among a plurality of sub-areas virtually divided into a space where an accident may occur in the industrial site. The processor may calculate an accident risk for each sub-area based on accident guide information for each of the plurality of sub-areas, and assign accident prevention rules to each sub-area based on the accident risk for each sub-area.

The processor identifies a sub-area in which the mobile robot is currently located among the plurality of sub-areas based on the log data, and remotely controls the mobile robot according to an accident prevention rule assigned to the identified sub-area. It can be configured to do so.

The processor grades the accident risk of each of the plurality of sub-areas into one of a plurality of sections, sets one of a plurality of safety distances corresponding one-to-one to the plurality of sections for each sub-area, and sets the identified It may be configured to remotely control the mobile robot from the sub-area according to accident prevention rules assigned to the identified sub-area, provided that another mobile robot or worker is located within a safety distance set in the identified sub-area. .

The processor identifies the current location of the worker based on the 2D image collected by the data communication unit and the location signal collected from the worker's communication terminal located within the industrial site, and selects the worker's current location among the plurality of sub-areas. It may be configured to identify the sub-area in which the worker is currently located and remotely control the worker's communication terminal according to accident prevention rules assigned to the identified sub-area.

A site management method according to another aspect of the present invention is executable by the site management server, and includes 2D images from a plurality of cameras deployed at the industrial site and log data from one or more mobile robots deployed at the industrial site - The log data includes the location, path, direction and collision level of the mobile robot over time - collecting, cumulatively recording the 2D image and the log data, the 2D image and the log data Based on this, it includes detecting the occurrence of an accident event within the industrial site and storing accident description information indicating the time and location at which the accident event occurred in the data recorder as an accident history.

According to at least one of the embodiments of the present invention, high-definition field snapshot images taken at various angles are input, log data for the field device of the mobile robot controlled by the terminal is recorded, and the snapshot image is converted into a 3D image. By performing rendering conversion, synchronized device status, and neural rendering image model to predict and prevent collision accidents between mobile robots and workers, the status of the site can be accurately identified using data obtained from cameras and mobile robots, and the situation can be monitored in detail remotely. By being able to see and continuously obtain data, you can have the effect of being able to prepare not only for the current situation but also for situations that will occur in the future.

In addition, according to at least one of the embodiments of the present invention, the occurrence of an accident is predicted from a 3D image, and guidance is provided to mobile robots and workers to prevent recurrence of accidents, so that if an accidental death occurs, the exact cause of the accident is identified and the situation at the time is determined. You will be able to understand accurately, thereby preventing similar accidents from occurring in the future, conducting remote training for employees using 3D digital twins, and increasing efficiency in industrial sites through training through models similar to those in the field. You can have

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention together with the detailed description of the invention described later, so the present invention includes the matters described in such drawings. It should not be interpreted as limited to only .
Figure 1 illustrates an accident situation at an industrial site.
Figure 2 is a schematic diagram schematically showing the configuration of an industrial site monitoring system according to an embodiment of the present invention.
FIG. 3 is a diagram referenced to explain an exemplary internal appearance of an industrial site to which the industrial site monitoring system shown in FIG. 1 is applied.
Figure 4 illustrates an image that reproduces the accident situation that occurred at the industrial site shown in Figure 3 in three-dimensional form.
FIG. 5 is a diagram illustrating the configuration of the field management server shown in FIG. 2.
Figure 6 is a flowchart illustrating an on-site management method according to an embodiment of the present invention.
Figure 7 is a flowchart illustrating an on-site management method according to another embodiment of the present invention.
Figure 8 is a flowchart illustrating an on-site management method according to another embodiment of the present invention.

Details regarding the purpose and technical configuration of the present invention and its operational effects will be more clearly understood by the following detailed description based on the drawings attached to the specification of the present invention. Embodiments according to the present invention will be described in detail with reference to the attached drawings.

The embodiments disclosed herein should not be construed or used as limiting the scope of the present invention. It is obvious to those skilled in the art that the description, including embodiments, of this specification has various applications. Accordingly, any embodiments described in the detailed description of the present invention are illustrative to better explain the present invention and are not intended to limit the scope of the present invention to the embodiments.

The functional blocks shown in the drawings and described below are only examples of possible implementations. Other functional blocks may be used in other implementations without departing from the spirit and scope of the detailed description. Additionally, although one or more functional blocks of the present invention are shown as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software components that perform the same function.

Terms containing ordinal numbers, such as first, second, etc., are used for the purpose of distinguishing one of the various components from the rest, and are not used to limit the components by such terms.

In addition, the expression including certain components is an “open” expression and simply refers to the presence of the corresponding components, and should not be understood as excluding additional components.

Furthermore, when a component is referred to as being “connected” or “connected” to another component, it should be understood that although it may be directly connected or connected to the other component, other components may exist in between. do.

Figure 2 is a schematic diagram schematically showing the configuration of an industrial site monitoring system according to an embodiment of the present invention.

Referring to FIG. 2, the industrial site monitoring system 100 includes a camera (C), a mobile robot (AGV), a site management server 200, and a communication terminal (T). Here, there may be multiple cameras (C), mobile robots (AGV), and communication terminals (T).

In industrial sites, multiple facilities (E) are placed at designated locations, and mobile robots (AGVs) and workers (U) ( As 64) moves, processes, tasks, or tasks that meet the purpose of the industrial site are carried out.

The camera C may be one or a combination of two or more of a monocular camera, a stereo camera, an infrared camera, or a CCTV.

Each mobile robot (AGV) may be referred to as an automated guided vehicle, a mobile robot, or an automated guided vehicle (AGV). A mobile robot (AGV) includes a driving unit that provides its own driving functions (e.g., electric motors and wheels), a communication circuit, and at least one type of spatial awareness sensor and situation detection sensor (e.g., shock sensor, speedometer, gyroscope, GPS module). ), and is provided to perform tasks (tasks) assigned to it through fully autonomous driving, partially autonomous driving, or fully controlled driving.

The field management server 200 is installed inside or outside the industrial site and is provided to interact with multiple cameras (C) and mobile robots (AGV) through a wired or wireless communication network.

The field management server 200 can remotely control the operation of multiple cameras (C) and mobile robots (AGV) located throughout the industrial site, and displays 2D images acquired for each camera (C) and mobile robots (C). Log data indicating the operating status of the AGV can be collected. The field management server 200 may store the collected data in its own storage 222 or in an external cloud. The cloud storage monitors the operation of the field management server 200 using log data received from the field management server 200 and responds to abnormalities so that the field management server 200 does not operate abnormally. Log data includes the position, collision level, movement path, speed, acceleration, direction, posture, and 2D image (acquired by the mobile robot) of the mobile robot (AGV).

The field management server 200 can convert the 2D image captured by the camera C into a 3D image. At this time, an artificial intelligence-based rendering algorithm such as Instant NeRF may be used to convert a 2D image into a 3D image.

The site management server 200 can calculate whether and/or the possibility of an accident occurring in an industrial site based on at least one of 2D images, 3D images, and log data, and further calculates the probability of an accident occurring at an industrial site, and further calculates the likelihood of an accident occurring at an industrial site based on at least one of 2D images, 3D images, and log data. ), guidance or remote control may be provided to prevent accidents from occurring for at least one of the following. In addition, if a safety accident actually occurs unavoidably, the field management server 200 automatically determines the time and cause of the accident, establishes measures to prevent recurrence of the same accident, and then has the worker (U) confirm the results. It can be returned in any possible form.

As an example, when log data collected from a mobile robot (AGV) includes a collision level above a certain level, the field management server 200 can identify (estimate) the time of accident based on the time code of the collision level. In addition, 2D images captured over a time period including the time of the accident can be identified and converted into 3D images. At this time, the length of the time section may be set to be proportional to the size of the collision level. In other words, an increasing collision level means that the severity of the accident is increasing, so if it is determined that a severe collision has occurred, the length of the time section for which 3D images are to be secured is determined as part of a detailed understanding of the circumstances before and after the accident. It increases that much.

Of course, the on-site management server 200 applies object recognition algorithms to 2D images or 3D images to enable recognition between mobile robots (AGVs), between mobile robots (AGVs) and workers (U), and between mobile robots (AGVs) without log data. It is also possible to detect and record when, where, and with what severity an accident occurred between the user and the facility (E) or between the worker (U).

The field management server 200 can convert 2D images collected by each camera (C) into 3D images in real time or near real time, regardless of whether an accident occurs within the industrial site, thereby creating a digital twin for the industrial site. This can be created.

A digital twin with a 3D video format is a real-time or near-real-time visualization of the entire or at least part of an industrial site captured by multiple cameras (C), or a portion of an industrial site captured by one or more cameras (C). It may be visualized in real time or near real time.

The on-site management server 200 may separately manage the above-mentioned time section, that is, the 3D image section in the accident occurrence time zone, as an accident reproduction image. This accident reproduction video is rendered in 3D form, and can be played repeatedly on the worker's (U) communication terminal (T) (e.g., AR glasses, head-mounted display, smartphone, personal computer, etc.) as well as through touch operations or gestures. It can be displayed as an angle-switched/enlarged/reduced view.

On the other hand, in industrial sites, as described above, a large number of facilities (E) are placed so that the mobile robot (AGV) and the worker (U) cannot move, and the remaining space, that is, the mobile robot (AGV) and the worker (U) ) can be divided into spaces where movement is possible. The space where the mobile robot (AGV) and the worker (U) can move can be set in the field management server 200 as a space where accidents can occur, and limited hardware and software resources can be used efficiently by monitoring only the spaces where accidents can occur. It can be operated.

A bird view (top view) defining a space where an accident can occur may be pre-produced and stored in the field management server 200. The target area of this bird view can be virtually divided into a number of sub-areas of a predetermined size, such as a grid, and each sub-area can be assigned a unique identification code in advance. If an accident occurs within an industrial site, the site management server 200 can map and store the accident guide information for the accident to the identification code of each sub-area where the accident occurred. As a result, accident history, which indicates which types of accidents frequently occur at which locations and at what times in industrial sites, can be precisely accumulated and managed, and not only can the probability of occurrence by accident type be predicted, but it is also possible to establish prevention measures for each accident type. . In addition, when a specific type of accident history is requested from the worker's (U) communication terminal (T), there is an advantage that only the recorded accident history corresponding to the corresponding type can be selected and immediate feedback is possible.

The field management server 200 predicts the possibility of an accident according to the current operating state of the mobile robot (AGV) based on past accident history information including accident events and guides, and when the possibility of an accident is high (when it exceeds a certain threshold) ) Mobile robots (AGVs) can be remotely controlled to safely avoid accidents. The field management server 200 monitors the possibility of an accident of the remotely controllable mobile robot (AGV), and if the possibility of an accident is high (exceeds a certain threshold), the field management server 200 reports the possibility of an accident of the mobile robot (AGV) that can be remotely controlled. You can command remote control.

The log data collected by the on-site management server 200 directly or indirectly describes the operation and surrounding situation of the mobile robot (AGV) not only at the time of the accident, but also for a certain period of time before and after the accident. For example, if the accident type is a collision between a mobile robot (AGV) and a worker (U) near a corner of an aisle that corresponds to a blind spot caused by equipment (E), the mobile robot (AGV) avoids the blind spot caused by the corner. Failure to check and reduce speed in advance is identified as the cause of the accident. A blind spot in an industrial site from the perspective of a mobile robot (AGV) refers to the original sensing range of the sensors (e.g. camera (C), lidar, radar, ultrasonic sensor, etc.) installed on the mobile robot (AGV) itself. It refers to an area or space that is restricted (blocked) by on-site facilities (E) or other obstacles.

The main functions of the field management server 200 largely include an accident reproduction function, an accident cause identification function, and an accident prevention function, and each function will be described in more detail with reference to the drawings below.

FIG. 3 is a diagram referenced to explain an exemplary internal appearance of an industrial site to which the industrial site monitoring system shown in FIG. 1 is applied.

Referring to Figure 3, in an industrial site, a number of facilities (E) are located at predetermined locations. Four cameras (C) are installed at different locations within the industrial site, each generating 2D images for a unique shooting range. For convenience of explanation, one mobile robot (AGV) and one worker (U) are shown. Compared to FIG. 1, in FIG. 3, a large number of cameras (C) are arranged, so that all spaces where accidents can occur within an industrial site can be recorded as video.

The movement path, location, and direction of each mobile robot (AGV) and worker (U) are captured by four cameras (C), and the field management server 200 processes the 2D images acquired for each camera (C) ( For example, by detecting the worker (U) and the robot as objects, it is possible to recognize the time of the collision accident between the mobile robot (AGV) and the worker (U), as well as the physical situation in the industrial site in the time period before and after the accident.

Just before a collision, the AGV continues to move because it is difficult to detect a worker (U) or another AGV approaching near the opposite corner due to the limited sensing range of the AGV itself. Despite the possibility of a collision accident, the AGV continues to move toward the corner due to the blind spot of the AGV, which may cause an accident where the AGV collides with the worker (U) near the corner. there is.

Figure 4 illustrates an image that reproduces the accident situation that occurred at the industrial site shown in Figure 3 in three-dimensional form.

Referring to FIG. 4, the field management server 200 can convert a 2D image into a 3D image, and as in the bottom situation of FIG. 3, a mobile robot (AGV) moves near a corner while moving between facilities (E). It shows a situation where a collision accident occurred between a robot (AGV) and a worker (U).

The industrial site monitoring system 100 according to the present invention is capable of specifically extracting only the 3D image of the area where the accident occurred out of the entire space of the industrial site, and for the purpose of preventing the occurrence of the same type of accident in the future, If the possibility of an accident exceeds the threshold, guidance is provided to at least one of the mobile robot (AGV) and the worker (U) to prevent recurrence of the accident.

The field management server 200 processes the 3D image obtained by converting the 2D image obtained for each camera (C) (e.g., detects the worker (U) and the robot as objects) to identify the mobile robot (AGV) and the worker (U). It is possible to recognize the physical situation within the industrial site at the time of the collision as well as before and after the accident.

FIG. 5 is a diagram illustrating the configuration of the field management server 200 shown in FIG. 2.

Referring to FIG. 5, the field management server 200 includes a data communication unit 210 and a controller 220.

The data communication unit 210 receives 2D images (eg, snapshot images) captured at various angles (capturing ranges) by two or more cameras (C). At this time, the 2D image collected from the camera C may be a high-definition image with a resolution of a certain level or higher.

The controller 220 includes an input/output interface 221, a data recording unit 222, and a processor 223, and data buses 224 and 134 connecting them to enable communication.

The input/output interface 221 transmits the sensing information from the industrial site collected from the data communication unit 210 to the data recording unit 222 and/or the processor 223 through the data bus 224, and the processor 223 performs the processing described later. Signals, data, information and/or commands generated by executing the methods are transmitted to the input/output interface 221.

The data recording unit 222 is hardware-wise: flash memory type, hard disk type, SSD type (Solid State Disk type), SDD type (Silicon Disk Drive type), and multimedia card micro type ( multimedia card micro type), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM) memory) may include at least one or two or more types of storage media. The data recording unit 222 may include a storage medium storing a computer program in which instructions for executing a site management method described later are recorded.

The data recording unit 222 stores learning models, computer programs, and/or data required for industrial site management according to the present invention. Additionally, the data recording unit 222 records 2D images for each camera C as well as log data collected from a mobile robot (AGV) that runs autonomously or is controlled by the field management server 200. In addition, the data recording unit 222 may also record log data including the location signal of the communication terminal (T) for each worker (U) collected by the data communication unit 210.

The processor 223 is operably coupled to the input/output interface 221 and the data recording unit 222 and controls the overall operation of the field management server 200. The processor 223 is hardware-wise, including application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and microprocessors ( 223) (microprocessors), and may include at least one electrical unit for performing other functions.

The processor 223 may include a main control unit 231, a cause analysis unit 232, and an accident prevention unit 233 as functional blocks.

The main control unit 231 not only converts the 2D image into a 3D image, but also synchronizes the time standards of the camera (C) and the mobile robot (AGV). In addition, the main control unit 231 can extract an accident reproduction image and update the accident history recorded in the data recorder 222 whenever an accident occurs.

The cause analysis unit 232 determines whether and/or the possibility of an accident (accident event) by type occurring to the mobile robot (AGV) and/or worker (U) based on at least one of log data, 2D image, and 3D image. can be determined and the cause of an accident event that has already occurred can be estimated (identified).

The accident prevention department (233) provides preemptive guidance (safety measures) to mobile robots (AGVs) and workers to prevent accidents that may occur in the future based on history information of past accident(s) within the industrial site. It can be provided to at least one of (U).

Figure 6 is a flowchart illustrating an on-site management method according to an embodiment of the present invention. The method of FIG. 6 may be for executing the accident reproduction function of the field management server 200. Each step of the method of FIG. 6 may be executed by the processor 223.

Referring to FIG. 6, in step S610, the site management server 200 collects sensing information about the industrial site. The collected sensing information may be recorded in a storage provided inside and/or outside the field management server 200. Here, the sensing information basically includes 2D images for each camera (C) and may further include log data for each mobile robot (AGV) and/or log data for each communication terminal (T).

In step S620, the site management server 200 determines whether an accident has occurred in the industrial site based on the sensing information. If the value of step S620 is “Yes,” the method of FIG. 6 proceeds to step S630.

In step S630, the field management server 200 extracts accident description information from the sensing information. The accident description information includes the object, time, location, intensity (e.g., collision intensity), and type (e.g., mobile robot (AGV)-only accident, operator (U)-only accident, mobile robot ( This may be information indicating accidents between AGVs, etc.

In step S640, the field management server 200 generates an accident reproduction image based on the accident description information. Specifically, the on-site management server 200 specifies the 2D image section corresponding to the accident time indicated by the accident description information, and then converts the specified 2D image into a 3D image. Additionally, the creation of an accident reproduction image can be completed by extracting the part corresponding to the accident location from the 3D image.

In step S650, the field management server 200 updates the accident history at the industrial site using the accident description information of the accident determined to have occurred in step S620.

In FIG. 6, step S650 is shown to follow step S640, but this is only an example, and step S640 may follow step S650, or one of the two steps may be executed in parallel while the other is executed.

Figure 7 is a flowchart illustrating an on-site management method according to another embodiment of the present invention. The method of FIG. 7 may be for executing the accident cause identification function of the field management server 200. Each step of the method of FIG. 7 may be executed by the processor 223.

Referring to FIG. 7, in step S710, the field management server 200 sets a cause identification target from the accident event list sorted in the accident history stored in the data recorder 222. For example, the field management server 200 may list accidents that have occurred in the past within an industrial site according to a predetermined standard (e.g., time of occurrence) and provide the list to the communication terminal (T) of the worker (U), and the worker (U) When a specific accident is selected from the accident list, the communication terminal (T) can notify the field management server 200 of the selected specific accident as a target for cause identification.

In step S720, the field management server 200 obtains accident description information of the accident event set as the cause identification target in step S710 from the data recorder 222.

In step S730, the field management server 200 inputs the accident description information (additionally 2D images or 3D images may be used together) obtained in step S720 into the cause analysis model to identify the cause from the cause analysis model. The accident cause of an accident event can be identified. Here, the cause analysis model may be a deep learning model pre-trained through a large learning data set, has multiple clusters mapped to different accident causes, and when any accident explanation information is received as input data, the corresponding Input data is classified into one cluster among the plurality of clusters. The field management server 200 outputs the cause of the accident mapped to a cluster classified by the operation of the cause analysis model as the correct answer to the accident explanation information obtained in step S720. As an example, the accident explanation information is (i) a specific mobile robot (AGV) sensed a collision exceeding a threshold near a corner at a specific time, (ii) a worker (U) and a specific mobile robot in a 2D image synchronized at the same time. If the (AGV) is recognized as an object located within a predetermined distance and (iii) the speed of at least one of the operator (U) and the specific mobile robot (AGV) rapidly decreases by more than a predetermined value at the same time, the corresponding accident event It can be identified that the cause is that a specific mobile robot (AGV) ran without reducing its speed in advance due to a blind spot near a corner.

In step S740, the field management server 200 may map the accident classification code previously assigned to the accident cause identified in step S730 to the accident event set as the cause identification target. Each accident cause can be categorized into several categories, and a unique accident classification code is assigned to each category, thereby facilitating the management and search of accident causes for each accident event by managers.

Figure 8 is a flowchart illustrating an on-site management method according to another embodiment of the present invention. The method of FIG. 8 may be for executing the accident prevention function of the field management server 200. The accident prevention function takes certain safety measures to prevent the same type of accident from occurring in the future based on past accident history. Each step of the method of FIG. 8 may be executed by the processor 223.

Referring to FIG. 8, in step S810, the site management server 200 provides accident guide information mapped for each sub-area of the space where accidents can occur predefined within the industrial site, based on the accident history stored in the data recorder 222. Acquire.

Accident guide information in any sub-area may be updated whenever an accident event occurs in the corresponding sub-area. Accident guide information mapped to an arbitrary sub-area includes the total number of accidents that have occurred in the past in that sub-area and the type of accident (e.g., collision between robots, collision between workers (U), collision between robots and workers (U)). It may indicate at least one of collision, collision between robot and facility, collision between worker (U) and facility) and occurrence time and severity (e.g., collision intensity, level of damage). If the accident guide information mapped to an arbitrary sub-area is the motion history (e.g. speed, movement path, etc.) of the mobile robot (AGV) over a certain period of time in the past based on the time of occurrence of the accident corresponding to the type related to the robot. It can be expressed additionally. In addition, the accident guide information mapped to an arbitrary sub-area may further indicate the cause of each accident that occurred in the corresponding sub-area.

In step S820, the field management server 200 calculates the accident risk for each sub-area based on the accident guide information mapped for each sub-area of the space where an accident can occur. Here, the accident risk may correspond to the total score for each item indicated by the accident guide information (e.g., has a proportional, positive correlation). For example, items in a certain sub-domain (e.g., total number of accidents, type of accident, occurrence time of each accident, and severity of each accident) can each be scored through pre-given calculation logic (e.g., function, etc.), and all items By adding up the scores, the accident risk of that sub-area can be determined. The calculation logic may be designed so that, among various accident types, the type in which a manager (person) is a party to the accident is assigned a higher score than the type of accident in which a manager (person) is not a party to the accident. In addition, the above calculation logic makes it difficult to take immediate safety measures due to a relatively low concentration of people or a small number of managers, such as at night, so accidents that occur at times within a predetermined time zone are more likely to occur than accidents that occur at other times. It may be designed to result in high scores being assigned.

In step S830, the field management server 200 assigns accident prevention rules to each sub-area based on the accident risk for each sub-area. For example, if the entire range of accident risk is divided (graded) into three sections (e.g., low, medium, high), the field management server 200 sets separate restrictions for the sub-area corresponding to the 'low' section. On the other hand, the first accident prevention rule (e.g. maximum speed limit) is given to the lower area corresponding to the 'medium' section, and the second accident prevention rule (e.g. , danger warning, route avoidance, and search for other routes) can be given. Grading into three sections is only an example, and two or more sections are acceptable.

In step S840, the field management server 200 remotely controls at least one of the mobile robot (AGV) and the communication terminal (T) using the accident prevention rules assigned to each sub-area in step S830. To this end, the field management server 200 continuously tracks (identifies) which of the plurality of sub-areas the current location of each mobile robot (AGV) is, and additionally continuously tracks the current location of each worker (U). .

As an example, the field management server 200 determines that the mobile robot (AGV) moves from the lower area corresponding to the 'low' section to the 'medium' section based on at least one of the log data, 2D image, and 3D image of the mobile robot (AGV). If it is confirmed that it has entered the corresponding sub-area, the maximum speed of the mobile robot (AGV) can be lowered from 1 m/s to 0.3 m/s according to the first accident prevention rule.

As another example, the on-site management server 200 allows the operator (U) to move from a sub-area corresponding to the 'low' or 'medium' section based on at least one of the log data, 2D image, and 3D image of the mobile robot (AGV). If it is confirmed that the lower area corresponding to the 'upper' section has been entered, a warning message can be sent to the communication terminal (T) of the worker (U) in accordance with the second accident prevention rule. The communication terminal (T) that has received the warning message can generate an alarm sound above a predetermined negative pressure and simultaneously output a route guidance screen to encourage movement from the current lower area to the lower area corresponding to the 'low' section. Here, the sub-area in which the worker (U) is located can be determined based on the location signal (e.g., GPS signal) of the communication terminal (T) carried by the worker (U) instead of the 2D image or 3D image.

As another example, the field management server 200 determines that the mobile robot (AGV) moves from the lower area corresponding to the 'middle' section to the 'upper' section based on at least one of the log data, 2D image, and 3D image of the mobile robot (AGV). If it is confirmed that it has entered the sub-area corresponding to , the shortest path through the 'medium' section or a combination of the two can be searched, and the mobile robot (AGV) can be remotely controlled to run on the discovered shortest path. For reference, each mobile robot (AGV) is assigned a unique mission and drives within the industrial site. The mobile robot (AGV) includes the destination according to its mission in the log data and records it on the site management server 200. Can be transmitted.

Meanwhile, the remote control of at least one of the mobile robot (AGV) and the communication terminal (T) executed in step S840 is performed between the two mobile robots (AGV) instead of being unconditionally executed according to the accident prevention rules assigned for each sub-area. It can be executed under the condition that the distance between two workers (U) and between the mobile robot (AGV) and the worker (U) is less than or equal to a predetermined safety distance. For example, even if the mobile robot (AGV) is located in the lower area corresponding to the 'up' section, the worker (U) closest to the mobile robot (AGV) is located at a position more than the above safety distance from the mobile robot (AGV). If present, the mobile robot (AGV) can perform its mission while traveling without any restrictions at the basic specified speed (e.g., up to 1 m/s).

At this time, the field management server 200 may set a safety distance for each sub-area corresponding to the accident risk of each of the plurality of sub-areas. For example, the safety distance set in the lower area corresponding to the 'high' section may be greater than the safety distance set in the lower area corresponding to the 'middle' section. The safety distance for each section (grade) according to the accident risk may be stored in advance in the data recorder 222 in the form of a look-up table, etc.

The communication terminal T may receive information related to one or more accident events from the field management server 200 and display the received information on a graphic interface. Accordingly, among all accident events that have occurred to date in industrial sites, accident reproduction images of specific accident events requested by workers or managers are displayed on the communication terminal (T), which has the advantage of being able to be used for remote safety training for workers. There is.

The present invention is not limited to the specific embodiments and application examples described above, and various modifications can be made by those skilled in the art without departing from the gist of the invention as claimed in the claims. Of course, these modified implementations should not be understood separately from the technical idea or outlook of the present invention.

In particular, configurations that implement the technical features of the present invention included in the block diagram and flow chart shown in the drawings attached to this specification represent logical boundaries between the configurations. However, according to an embodiment of the software or hardware, the depicted configurations and their functions are executed in the form of stand-alone software modules, monolithic software structures, codes, services, and combinations thereof, and can execute stored program code, instructions, etc. Since the functions can be implemented by being stored in a medium executable on a computer equipped with a processor, all of these embodiments should also be regarded as falling within the scope of the present invention.

Accordingly, although the attached drawings and their descriptions illustrate the technical features of the present invention, they should not be simply inferred unless a specific arrangement of software for implementing these technical features is clearly stated. In other words, various embodiments described above may exist, and since such embodiments may be partially modified while retaining the same technical features as the present invention, these should also be regarded as falling within the scope of the present invention.

In addition, in the case of a flowchart, operations are depicted in the drawing in a specific order, but this is shown to obtain the most desirable results, and such operations must be executed in the specific order or sequential order shown, or all illustrated operations must be executed. It should not be understood as something that must be done. In certain cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and the program components and systems described are generally integrated together into a single software product or integrated into multiple software products. It should be understood that it can be packaged.

100: Industrial site monitoring system
C: Camera AGV: Mobile robot
U: worker T: communication terminal
E: Equipment
200: On-site management server
210: data communication unit 220: controller
221: input/output interface 222: data recorder
223: processor

Claims (8)

2D images from a plurality of cameras deployed at an industrial site and log data from one or more mobile robots deployed at the industrial site - the log data records the location, path, direction, and collision level of the mobile robot over time. Includes - Data and communications department that collects;
a data recording unit that cumulatively records the 2D image and the log data; and
A processor operably coupled to the data communication unit and the data recording unit,
The processor,
Based on the 2D image and the log data, detect the occurrence of an accident event within the industrial site,
Characterized in that the on-site management server is configured to store accident description information indicating the time and location at which the accident event occurred in the data recorder as an accident history.
According to paragraph 1,
The processor,
When the occurrence of the accident event is detected, the 2D image is rendered into a 3D image based on the accident description information of the accident event, and is configured to generate an accident reproduction image for at least a portion of the industrial site. , field management server.
According to paragraph 1,
The processor,
Input the accident explanation information of the accident event into a cause analysis model, wherein the cause analysis model is pre-trained to classify the accident explanation information into one of a plurality of clusters mapped to different accident causes,
Characterized in that the on-site management server is configured to map an accident classification code previously assigned to an accident cause mapped to a cluster classified by the cause analysis model with respect to the accident explanation information to the accident event.
According to paragraph 1,
The processor,
Update the accident guide information of the sub-area where the accident event occurred among a plurality of sub-areas virtually divided into the space where an accident may occur in the industrial site,
Based on the accident guide information for each of the multiple sub-areas, the accident risk for each sub-area is calculated,
An on-site management server characterized in that it assigns accident prevention rules to each sub-area based on the accident risk for each sub-area.
According to clause 4,
The processor,
Based on the log data, identify a sub-area in which the mobile robot is currently located among the plurality of sub-areas,
An on-site management server, characterized in that it is configured to remotely control the mobile robot according to accident prevention rules assigned to the identified sub-area.
According to clause 5,
The processor,
The accident risk of each of the plurality of sub-areas is graded into one of a plurality of sections,
Setting one of a plurality of safety distances corresponding one-to-one to the plurality of sections for each sub-area,
configured to remotely control the mobile robot from the identified sub-area according to accident prevention rules assigned to the identified sub-area, provided that another mobile robot or worker is located within a safety distance set in the identified sub-area. An on-site management server, characterized in that
According to clause 4,
The processor,
Identifying the current location of the worker based on the 2D image collected by the data communication unit and the location signal collected from the worker's communication terminal located in the industrial site,
Identifying the sub-area in which the worker is currently located among the plurality of sub-areas,
An on-site management server, characterized in that it is configured to remotely control the worker's communication terminal according to accident prevention rules assigned to the identified sub-area.
In the field management method executable by the field management server according to any one of claims 1 to 8,
2D images from a plurality of cameras deployed at the industrial site and log data from one or more mobile robots deployed at the industrial site - the log data includes the location, path, direction, and collision level of the mobile robot over time Includes - collecting;
Accumulatingly recording the 2D image and the log data;
Detecting the occurrence of an accident event within the industrial site based on the 2D image and the log data; and
Storing accident description information indicating the time and location at which the accident event occurred in the data recorder as an accident history;
An on-site management method comprising:

Images