KR20240123442A - Mobile sterilization-robot with human pose estimation - Google Patents

Abstract

The present invention relates to a mobile sterilization robot with applied posture recognition technology, which moves to a sterilization-required area, which is a place requiring sterilization, through autonomous driving, and at the same time, upon arrival at the sterilization-required area, detects a human body and performs sterilization work using a UVC sterilization unit, a UVA sterilization unit, and a plasma ion unit depending on whether a human body is detected, and detects the pose of a detection object using HPE (Human Pose Estimation) to which a PoseNet model is applied, and classifies the detection object into ‘temporary mode’ or ‘visit mode’ based on the detected pose and determines whether to move to another place or wait based on the classification result, thereby maximizing the sterilization effect and efficiency of a wide area without human intervention, and minimizing human harm by increasing the accuracy of human body detection, and effectively preventing the spread of viruses and bacteria, and capable of maintaining the sterilization effect even in a space densely populated with a critical number of visitors.

Description

{Mobile sterilization-robot with human pose estimation}

The present invention relates to a mobile sterilization robot to which posture recognition technology is applied, and more specifically, to a mobile sterilization robot that autonomously moves to a sterilization-required area, which is a place requiring sterilization, and at the same time, upon arrival at the sterilization-required area, detects a human body and performs sterilization work using a UVC sterilization unit, a UVA sterilization unit, and a plasma ion unit, and detects the pose of a detection object using HPE (Human Pose Estimation) to which a PoseNet model is applied, and classifies the detection object into a ‘temporary mode’ or a ‘visit mode’ according to the detected pose, and determines whether to move to another place or wait according to the classification result, thereby maximizing the sterilization effect and efficiency of a wide area without human intervention, and minimizing harm to the human body by increasing the accuracy of human body detection, and effectively preventing the spread of viruses and bacteria, and is capable of maintaining a sterilization effect even in a space densely populated with a critical number of visitors.

In general, mobile robots are service robots that can replace or help with difficult and hard work in the space we live in. For example, mobile robots determine the distance to obstacles installed in the area through various sensors without user intervention, and change direction on their own to perform given tasks.

In other words, mobile robots are widely used to perform tasks that are complex in structure and dangerous for users to perform directly, and the demand for them is expected to gradually increase.

Meanwhile, as various infectious diseases that can be transmitted through humans or livestock, such as COVID-19, Middle East Respiratory Syndrome (MERS), Severe Acute Respiratory Syndrome (SARS), and avian influenza (AI), have recently appeared, interest in quarantine systems to prevent the spread of infectious diseases is rapidly increasing.

These infectious diseases can spread to people around them through various infection routes, but the main route of infection is through the respiratory tract.

In particular, indoor spaces have the disadvantage of being vulnerable to the spread of viruses, bacteria, and germs because they are enclosed areas where various people reside and visit.

Accordingly, various studies are being conducted to periodically sterilize and disinfect the internal air of indoor spaces, and research using mobile robots is actively being conducted as one of the methods.

Figure 1 is a block diagram showing an ultraviolet sterilizing mobile robot disclosed in domestically registered patent No. 10-1742489 (Title of invention: Ultraviolet sterilizing mobile robot device and system).

The ultraviolet sterilizing mobile robot (hereinafter referred to as “prior art”) (100) of Fig. 1 is composed of a communication unit (110) that receives a control signal from an external device, a sensor unit (120) that detects external terrain and a distance from an external object, a power unit (130) that supplies power, an ultraviolet sterilizing unit (150) that irradiates ultraviolet rays to the outside, a display unit (160) that displays the current status, a power charging unit (170) that charges power from an external power source, and a control unit (140) that controls the movement path using data detected by the sensor unit (120) and controls ultraviolet sterilization of the ultraviolet sterilizing unit (150).

If the sensor unit (120) does not detect a living organism while the operation of the ultraviolet sterilization unit (150) is stopped, the control unit (140) restarts the operation of the ultraviolet sterilization unit (150). However, if the sensor unit (120) detects a living organism for a certain period of time while the operation of the ultraviolet sterilization unit (150) is stopped, the control unit (140) instructs a change in the movement path to an area where the living organism is not detected.

The prior art (100) configured in this way has the advantage of increasing safety by preventing people from being exposed to ultraviolet rays in advance since ultraviolet irradiation is determined based on whether or not the human body is detected.

However, since the sensor unit (120) of the prior art (100) uses a known human body detection sensor such as a PIR (Passive infrared) sensor or a microwave motion sensor, it has a structural limitation in that it cannot simply determine whether a human body has been detected.

In general, indoor spaces, if the number of residents or visitors is small, are efficiently configured to perform ultraviolet sterilization only by detecting human bodies. However, if the number of residents or visitors is large, the detected human bodies have the characteristic of being mixed between people who have temporarily visited the area and people who will stay in the area for a certain period of time.

That is, considering the characteristics of an indoor location, if it is determined that the detected object is a person who has temporarily visited the area, the sterilization operation can be performed after waiting until no human bodies are detected without moving to another area, but since the prior art (100) does not consider the characteristics of such indoor locations at all, in the case of an indoor location where residents or visitors are densely packed, there is a structural limitation in that a human body is detected in every area moved to, making it impossible to perform the sterilization operation.

The present invention is intended to solve such problems, and the solution of the present invention is to provide a mobile sterilization robot that can maximize sterilization effects and efficiency by moving to a place requiring sterilization and performing sterilization work autonomously without human intervention, thereby reducing unnecessary manpower consumption.

In addition, another problem of the present invention is to provide a mobile sterilization robot configured to perform sterilization work only when a human body is not detected, after moving to a destination area (S’), without performing sterilization work, thereby preventing exposure of the human body to the sterilization work, thereby enhancing service safety and reliability.

In addition, another problem solved by the present invention is to provide a mobile sterilization robot that can further increase the sterilization effect and efficiency compared to the same sterilization working time by providing sterilization functions of various functions including a UVC unit, a UVA unit, and a plasma ion unit.

In addition, another problem of the present invention is to provide a mobile sterilization robot that can dramatically solve the conventional problem of sterilization work not being possible when people are crowded in an indoor space, by configuring that the HPE-based analysis unit of the controller not only detects a human body, but also detects the pose of the detected human body, and then sets the state mode to ‘temporary mode’ or ‘visit mode’, and the standby or not determination unit determines whether to standby or move according to the state mode of the detected object even if the detected object exists.

The present invention provides a solution for the above-described problem, comprising: a camera, a sterilization module, a moving unit, and a controller, wherein the mobile sterilization robot autonomously moves to areas (S’) of an indoor space (S) and sterilizes the air in the area (S’) to which it has moved; wherein the controller estimates its own position, detects the position of the mobile sterilization robot on a map to generate a driving path, and controls the moving unit to move along the generated driving path; an image collection unit that operates the camera when the movement by the autonomous driving unit is completed, and collects an image acquired by the shooting of the camera; an HPE-based analysis unit that analyzes the image collected by the image collection unit to detect a human body (object); and a sterilization-work determination unit that determines that it is okay to proceed with the sterilization work if 1) the detected object in the HPE-based analysis unit is ‘0’, but 2) determines that the sterilization work cannot be performed if the detected object is ‘1’ or more. It is executed when the above sterilization-work judgment unit determines that it is okay to proceed with the sterilization work, and includes a sterilization-work operation unit that operates the sterilization module to proceed with the sterilization work.

In addition, in the present invention, the HPE-based analysis unit analyzes the image collected by the image collection unit to detect a human body (object) and at the same time detects the pose of the detected object using a pre-learned pose detection algorithm, determines the status mode of the pose of the detected object as ‘temporary mode’ or ‘visit mode’, and the controller is executed when the sterilization-work determination unit determines that the sterilization work cannot be performed, and if 1) there is an object whose status mode is ‘visit mode’ among the detected objects, it is finally determined not to perform the sterilization work, but 2) if there is no object whose status mode is ‘visit mode’, it is determined to wait, and when a preset waiting time (t) has elapsed, it is preferable to further include a standby determination unit that restarts the image collection unit, the HPE-based analysis unit, and the sterilization-work determination unit.

In addition, in the present invention, the standby determination unit includes: a state-mode input module that receives state-mode information of each object from the HPE-based analysis unit; a visit mode object search module that searches for the state-mode information of each object input through the state-mode input module to search whether an object whose state-mode is a visit mode exists among the objects; a movement decision module that is executed when an object having a state-mode of the visit mode exists among the objects in the visit mode object search module, and after the mobile sterilization robot decides not to wait in the corresponding area (S’) but to move to the next area, outputs a movement command signal to the autonomous driving unit; a standby decision module that is executed when an object having a state-mode of the visit mode does not exist among the objects in the visit mode object search module, and determines that the mobile sterilization robot will wait for a preset standby time (t) in the corresponding area (S’); an elapsed time measurement module that is executed when the standby decision module determines to wait, and measures an elapsed time (△t) using a timer; An alarm display module that is executed when the above-mentioned standby decision module determines to wait and displays alarm information to the outside; a standby time elapsed confirmation module that compares whether the elapsed time (△t) measured by the above-mentioned elapsed time measurement module has elapsed the standby time (t) and outputs a control signal to the image collection unit (34) when the elapsed time (△t) becomes the standby time (t), and when the image collection unit receives a control signal from the standby time elapsed confirmation module, it is preferable that the image collection unit repeats the previous process, and when the above-mentioned autonomous driving unit receives a movement command signal from the above-mentioned movement decision module, it selects the next zone (S’) and then controls the movement unit to move to the next zone (S’).

In addition, in the present invention, the controller further includes a memory storing matching information in which pose types belonging to the 'temporary mode' and pose types belonging to the 'visit mode' are matched, and it is preferable that the HPE-based analysis unit includes an image analysis module that analyzes an image collected by the image collection unit; a human body detection module that detects human bodies in an image by utilizing analysis data detected by the image analysis module; a PoseNet-based object joint detection module that uses the image of each object detected by the human body detection module as input data of a PoseNet model and outputs joint points of each object; an object-specific pose detection module that analyzes the joint points of each object detected by the PoseNet-based object joint detection module and detects the pose of each object; and a state-mode setting module that sets whether the state-mode of each object is the 'temporary mode' or the 'visit mode' by referring to the matching information stored in the memory and the pose information of each object detected by the object-specific pose detection module.

In addition, in the present invention, it is preferable that the sterilizing module includes a plasma ion unit installed on the path of the inhaled air and generating plasma ions; a UVA sterilizing unit installed on the path of the inhaled air and irradiating UVA; and a UVC sterilizing unit irradiating UVC to the corresponding area (S’).

According to the present invention having the above-mentioned tasks and solutions, the sterilization work can be performed autonomously to a location requiring sterilization without human intervention, thereby maximizing the sterilization effect and efficiency, and reducing unnecessary manpower consumption.

In addition, according to the present invention, after moving to the destination area (S’), if a human body is detected, the sterilization operation is not performed, but is performed only if a human body is not detected, so that the sterilization operation is not exposed to the human body, thereby increasing service safety and reliability.

In addition, according to the present invention, the sterilization unit provides sterilization functions of various functions including a UVC unit, a UVA unit, and a plasma ion unit, thereby further increasing the sterilization effect and efficiency compared to the same sterilization-working time.

In addition, according to the present invention, the HPE-based analysis unit of the controller not only detects a human body, but also detects the pose of the detected human body, and then sets the status mode to ‘temporary mode’ or ‘visit mode,’ and the standby or not determination unit is configured to determine whether to standby or move based on the status mode of the detected object even if the detected object exists, thereby dramatically resolving the conventional problem of sterilization work not being possible when people are crowded in an indoor space.

Figure 1 is a block diagram showing an ultraviolet sterilizing mobile robot disclosed in domestically registered patent No. 10-1742489 (Title of invention: Ultraviolet sterilizing mobile robot device and system).
Figure 2 is a block diagram showing a mobile sterilization robot to which posture recognition technology, which is an embodiment of the present invention, is applied.
Figure 3 is an example diagram showing Figure 2.
Figure 4 is another example of Figure 3.
Figure 5 is a block diagram showing the controller of Figure 2.
Figure 6 is a block diagram showing the HPE-based analysis unit of Figure 5.
Figure 7 is a conceptual diagram explaining HPE applied to Figure 6.
Figure 8 is a conceptual diagram showing the PoseNet model applied to HPE of Figure 7.
Figure 9 is a block diagram showing the standby judgment unit of Figure 5.

Hereinafter, an embodiment of the present invention will be described with reference to the attached drawings.

FIG. 2 is a block diagram showing a mobile sterilization robot to which posture recognition technology, which is an embodiment of the present invention, is applied, FIG. 3 is an exemplary diagram showing FIG. 2, and FIG. 4 is another exemplary diagram of FIG. 3.

The mobile sterilization robot (1) to which the posture recognition technology of one embodiment of the present invention of FIGS. 2 to 4 is applied moves to a sterilization-required area, which is a place requiring sterilization, through autonomous driving, and at the same time, upon arrival at the sterilization-required area, detects a human body and performs sterilization work using a UVC sterilization unit, a UVA sterilization unit, and a plasma ion unit, and detects the pose of a detection object using HPE (Human Pose Estimation) to which a PoseNet model is applied, and then classifies the detection object into a ‘temporary mode’ or a ‘visit mode’ according to the detected pose and determines whether to move to another place or wait according to the classification result, thereby maximizing the sterilization effect and efficiency of a wide area without human intervention, and minimizing harmfulness to the human body by increasing the accuracy of human body detection, and effectively preventing the spread of viruses and bacteria, and maintaining the sterilization effect even in a space where a critical number of visitors are densely packed.

In addition, the mobile sterilization robot (1) to which the posture recognition technology of the present invention is applied is composed of a controller (3), a suction fan (11), an exhaust fan (12), a plasma ion unit (13), a UVA sterilization unit (14), a UVC sterilization unit (15), a filter unit (16), a sensor unit (17), a display panel (18), an air quality sensor (19), and a moving unit (20), as shown in FIGS. 2 to 4.

The suction fan (11) sucks outside air, which is air from the area (S’) through which the mobile sterilization robot (1) has moved, into the interior of the body (10).

The exhaust fan (12) sucks in air into the interior of the body (10) through the suction fan (11) and then discharges the air, which is sterilized by the plasma ion unit (13), UVA sterilization unit (14), UVC sterilization unit (15) and filter unit (16), to the outside.

The plasma ion unit (13) is installed on the path of the inhaled air and generates plasma ions to sterilize the inhaled air.

The UVA sterilization unit (14) is installed on the air movement path and sterilizes the moving air by irradiating it with UVA.

The UVC sterilization unit (15) irradiates UVC to the area (S’).

The filter unit (16) is installed on the air movement path to filter the inflowing air, and more specifically, it is preferable to consist of a pre-filter, a HEPA filter, and an activated carbon filter.

The sensor unit (17) is composed of a camera that acquires high-resolution images. At this time, the images acquired by shooting by the camera, which is the sensor unit (17), are used to detect objects and estimate the pose of the detected object in the controller (3) of Fig. 5 described later.

The display panel (18) is installed on the front of the body (10) and displays the current air quality status, sterilization operation status, guidance text, and warning text.

The air quality sensor (19) senses the air quality of the internal air of the moved area (S’) and outputs the detected measurement value to the controller (3).

The moving part (20) is installed at the lower part of the body (10) and moves the body (10) in a horizontal direction under the control of the controller (3).

Figure 5 is a block diagram showing the controller of Figure 2.

The controller (3) of Fig. 5 is installed inside the body (10) of the mobile sterilization robot (1) and manages and controls the operations of the aforementioned components (11), (12), (13), (14), (15), (16), (17), (18), (19), and (20).

In addition, the controller (3) is composed of a control unit (30), a memory (31), a communication interface unit (32), an autonomous driving unit (33), an image collection unit (34), an HPE-based analysis unit (35), a sterilization-work judgment unit (36), a standby judgment unit (37), a sterilization-work operation unit (38), a sterilization-work information generation unit (39), and a display unit (40), as shown in FIG. 5.

The control unit (30) is an O.S (Operating System) of the controller (3) and manages and controls the operations of the control targets (31), (32), (33), (34), (35), (36), (37), (38), (39), and (40).

In addition, when the control unit (30) moves to the destination by the autonomous driving unit (33), it executes the image collection unit (34).

In addition, the control unit (30) executes the sterilization operation unit (38) if the sterilization operation judgment unit (36) determines that 1) the sterilization operation can proceed, but executes the standby judgment unit (37) if 2) the sterilization operation cannot proceed.

In addition, when the sterilization operation is completed by the sterilization operation operation unit (38), the control unit (30) executes the sterilization operation information generation unit (39), and when the sterilization operation information is generated by the sterilization operation information generation unit (39), the generated sterilization operation information is stored in the memory (31).

In the memory (31), the location information of the indoor location (S) where the mobile sterilization robot (1) is installed and the location information of each zone (S’) constituting the indoor location (S) are preset and stored.

In addition, an indoor map, which is an electronic map corresponding to the indoor location (S), is created and stored in the memory (31).

Additionally, the pre-trained PoseNet model is stored in memory (31).

At this time, the PoseNet model is an algorithm that analyzes the input video image and outputs joint (skeleton) data.

Additionally, destination information generated by the autonomous driving unit (33) is temporarily stored in the memory (31).

In addition, the memory (31) stores the sterilization-operation information generated by the sterilization-operation information generation unit (39).

The autonomous driving unit (33) controls the moving unit (20) so that autonomous driving is performed based on the location of the mobile sterilization robot (1) and the location information of the mobile sterilization robot (1) on the indoor map by utilizing scanning information generated by various sensors and a previously produced indoor map.

At this time, the autonomous driving unit (33) analyzes and utilizes detection data and scanning information acquired through various sensors, generates an indoor map through SLAM (Simultaneous localization and mapping) technology, estimates its own location, detects the location of the mobile sterilization robot (1) on the indoor map, generates a driving path, and then controls the moving unit (20) according to the generated driving path.

At this time, SLAM refers to a method of predicting the absolute position of an AI robot (1) in space by using its relative position value and the surrounding environment.

In addition, when the autonomous driving unit (33) completes moving to the destination area (S’), it outputs a control signal to the image collection unit (34).

The image collection unit (34) is executed when the mobile sterilization robot (1) is moved to the target area (S’) by the autonomous driving unit (33), and controls the sensor unit (17) so that the sensor unit (17) operates.

Additionally, the image collection unit (34) collects high-resolution images acquired by the camera, which is the sensor unit (17).

At this time, the high-resolution image collected by the image collection unit (34) is input to the HPE-based analysis unit (35) under the control of the control unit (30).

FIG. 6 is a block diagram showing the HPE-based analysis unit of FIG. 5, FIG. 7 is a conceptual diagram explaining the HPE applied to FIG. 6, and FIG. 8 is a conceptual diagram showing the PoseNet model applied to the HPE of FIG. 7.

The HPE-based analysis unit (35) of Fig. 6 is a processor that analyzes a high-resolution image acquired by shooting of the sensor unit (17), detects a human body (object), estimates the pose of the detected object at the same time, and classifies the detected object into ‘temporary mode’ or ‘visit mode’ according to the estimated pose.

In addition, the HPE-based analysis unit (35) is composed of an image analysis module (351), a human body detection module (352), a PoseNet-based first object joint detection module (353-1), ..., a PoseNet-based Nth object joint detection module (353-N), an object-specific pose detection module (354), and a state-mode setting module (355), as illustrated in FIG. 6.

The image analysis module (351) analyzes the image collected by the image collection unit (34), and the human body detection module (352) detects human bodies (1, ..., N) in the image by utilizing the analysis data detected by the image analysis module (351).

At this time, the control unit (30) terminates the operation of the HPE-based analysis unit (35) and executes the sterilization-work judgment unit (36) if 1) the detected human body is ‘0’ in the human body detection module (352), but 2) if the detected human body is ‘1’ or higher, executes the PoseNet-based first object joint detection module (353-1), ..., PoseNet-based Nth object joint detection module (353-N).

The first object joint detection module (353-1) based on PoseNet uses the image of the first object detected by the human detection module (352) as input data of the PoseNet model and outputs joint points of the first object.

In general, HPE (Human Pose Estimation) refers to a model that predicts human joints (skeletons) from images or videos, and as shown in (a) of Fig. 7, it operates in a top-down manner by first extracting a human image, and then estimating joints from the extracted human image, as shown in (b) of Fig. 7.

At this time, PoseNet can be implemented using ResNet and MobileNet models. Since ResNet has a larger CNN model size than mobileNet, more accurate pose estimation results can be expected. However, it may not be suitable for real-time operation due to the delay in page load time/inference time. Therefore, real-time implementation was made possible through mobileNet.

In addition, the PoseNet model obtains an output image of the same size as the original image (input image) without changing the size of the input image, and as shown in Figure 8, it can determine how precisely to view the input image by using a hyperparameter called output stride.

The PoseNet-based Nth object joint detection module (353-N) uses the image of the Nth object detected by the human detection module (352) as input data for the PoseNet model and outputs joint points of the Nth object.

The object-specific pose detection module (354) analyzes the joint points of each object detected by the PoseNet-based first object joint detection module (353-1), ..., PoseNet-based Nth object joint detection module (353-N) to detect the pose of each object.

At this time, the object-specific pose detection module (354) can determine the pose of each object by utilizing a reference table in which the position information of joint points for each preset pose is matched.

The state-mode setting module (355) sets whether the state-mode of each object is ‘temporary mode’ or ‘visit mode’ by referring to the pose information of each object detected by the object-specific pose detection module (354).

At this time, the temporary mode refers to poses that can be judged to have temporarily visited the area (S’), and the visit mode refers to poses that can be judged to have stayed in the area (S’) for a certain period of time or longer.

In addition, the state-mode setting module (355) can set the state-mode of each object by utilizing matching information in which pose types belonging to the ‘temporary mode’ and pose types belonging to the ‘visit mode’ are matched.

Returning to Fig. 5 again, looking at the sterilization-work judgment unit (36), the sterilization-work judgment unit (36) determines that 1) if the human body detected by the HPE-based analysis unit (35) is ‘0’, it is okay to proceed with the sterilization-work, and 2) if the human body detected by the HPE-based analysis unit (35) is ‘1’ or higher, it determines that the sterilization-work cannot be performed.

At this time, the control unit (30) executes the sterilization operation unit (38) if the sterilization operation judgment unit (36) determines that 1) the sterilization operation can proceed, but executes the standby judgment unit (37) if 2) the sterilization operation cannot proceed.

In other words, the sterilization-work judgment unit (36) determines not to perform sterilization work when a human body is detected within the visit area (S’), thereby preventing in advance any damage or safety accidents caused by UVC that is harmful to the human body.

Figure 9 is a block diagram showing the standby judgment unit of Figure 5.

The standby judgment unit (37) of Fig. 9 is executed under the control of the control unit (30) when the sterilization operation judgment unit (36) determines that the sterilization operation cannot be performed.

In addition, the standby judgment unit (37) is composed of a state-mode input module (371), a visit mode object search module (372), a movement decision module (373), a standby decision module (374), an elapsed time measurement module (375), an alarm display module (376), and a standby time elapsed confirmation module (377), as illustrated in FIG. 9.

The state-mode input module (371) receives the state-mode information of each object set in the state-mode setting module (355) of Fig. 6 described above.

The visit mode object search module (372) searches for the state-mode information of each object input through the state-mode input module (371), and more specifically, searches for whether an object having the state-mode of the visit mode exists among the objects.

At this time, the control unit (30) executes the movement decision module (373) in the visit mode object search module (372) if 1) an object having the visit mode status-mode exists among the objects, but 2) if an object having the visit mode status-mode does not exist among the objects, it executes the standby decision module (374).

The movement decision module (373) is executed when an object having a visit mode status-mode exists among the objects in the visit mode object search module (372), and the mobile sterilization robot (1) decides not to wait in the corresponding area (S’) but to move to the next area, and outputs a movement command signal to the autonomous driving unit (33).

At this time, when the autonomous driving unit (33) receives a movement command signal from the movement decision module (373), it selects the next zone (S’) and then controls the movement unit (20) to move to the next zone (S’).

The standby decision module (374) is executed when, among the objects in the visit mode object search module (372), there is no object with a visit mode status-mode, and the mobile sterilization robot (1) decides to wait for a preset standby time (t) in the corresponding area (S’).

The elapsed time measurement module (375) is executed when the standby decision module (374) determines that the mobile sterilization robot (1) is to standby, and measures the elapsed time (△t) using a timer.

The alarm display module (376) is executed when the standby decision module (374) determines to put the mobile sterilization robot (1) on standby, and uses a display panel (18), a speaker (not shown), and an LED (not shown) to display alarm information externally so that people located in the corresponding area (S’) are aware that sterilization work will be performed soon through the alarm information.

The waiting time elapsed confirmation module (377) compares whether the elapsed time (△t) measured by the elapsed time measuring module (374) has elapsed the waiting time (t), and if the elapsed time (△t) becomes the waiting time (t), it confirms that the waiting time has elapsed and then outputs a control signal to the image collection unit (34).

At this time, when a control signal is input to the image collection unit (34) by the waiting time elapsed confirmation module (377), the image collection unit (34), HPE-based analysis unit (35), sterilization-work judgment unit (36), and waiting judgment unit (37) are repeatedly executed.

The sterilization operation unit (38) is executed under the control of the control unit (30) when the sterilization operation judgment unit (36) determines that it is okay to proceed with the sterilization operation because the human body detected by the HPE-based analysis unit (35) is ‘0’.

In addition, the sterilization operation unit (38) operates the suction fan (11), exhaust fan (12), plasma ion unit (13), UVA sterilization unit (14), UVC sterilization unit (15) and filter unit (16) of Fig. 2 described above to perform sterilization operation in the corresponding area (S’).

The display section (40) displays sterilization operation status, air quality status, alarm information, guidance text, etc. through the display panel (18).

In this way, the mobile sterilization robot (1), which is one embodiment of the present invention, can move to a place requiring sterilization and perform sterilization work autonomously without human intervention, thereby maximizing the sterilization effect and efficiency, and reducing unnecessary labor consumption.

In addition, the mobile sterilization robot (1) of the present invention is configured to not perform sterilization work when a human body is detected after moving to the destination area (S’), but to perform sterilization work only when a human body is not detected, thereby preventing exposure of the human body during sterilization work, thereby increasing service safety and reliability.

In addition, the mobile sterilization robot (1) of the present invention provides sterilization functions of various functions including a sterilization unit, a UVC unit, a UVA unit, and a plasma ion unit, thereby further increasing the sterilization effect and efficiency compared to the same sterilization working time.

In addition, the mobile sterilization robot (1) of the present invention is configured so that the HPE-based analysis section of the controller not only detects a human body, but also detects the pose of the detected human body, and then sets the status mode to ‘temporary mode’ or ‘visit mode’, and the standby or not determination section determines whether to standby or move according to the status mode of the detected object even if a detected object exists, thereby dramatically resolving the conventional problem of sterilization work not being able to be performed when people are crowded in an indoor space.

1: Mobile sterilization robot with posture recognition technology
3:Controller 10:Body
11: Suction fan 12: Exhaust fan
13: Plasma ion unit 14: UVA sterilization unit
15: UVC sterilization unit 16: Filter unit
17: Sensor section 18: Display panel
19: Air quality sensor 20: Moving part
30:Control unit 31:Memory
32: Communication interface section 33: Autonomous driving section
34: Image collection unit 35: HPE-based analysis unit
36: Sterilization-Work Decision Department 37: Standby Decision Department
38: Sterilization-Work Operation Department 39: Sterilization-Work Information Generation Department
40: Display section 351: Image analysis module
352: Human detection module
353-1: PoseNet-based first object joint detection module
353-N:PoseNet-based Nth object joint detection module
354: Object-specific pose detection module 355: State-mode setting module
371: State-mode input module 372: Visit mode object search module
373:Movement Decision Module 374:Standby Decision Module
375: Elapsed time measurement module 376: Alarm display module
377: Waiting time elapsed check module

Claims (5)

A mobile sterilization robot comprising a camera, a sterilization module, a moving part, and a controller, which autonomously moves to areas (S') in an indoor space (S) and sterilizes the air in the area (S') to which it has moved:
The above controller
An autonomous driving unit that estimates its own location, detects the location of the mobile sterilization robot on a map, generates a driving path, and then controls the moving unit to move along the generated driving path;
When movement is completed by the autonomous driving unit, an image collection unit that operates the camera and collects images obtained by filming by the camera;
An HPE-based analysis unit that analyzes images collected by the above image collection unit to detect a human body (object);
In the HPE-based analysis unit above, 1) if the detected object is '0', it is determined that it is okay to proceed with the sterilization operation, but 2) if the detected object is '1' or higher, the sterilization operation judgment unit determines that the sterilization operation cannot be performed;
A mobile sterilization robot characterized by including a sterilization operation unit that is executed when the sterilization operation judgment unit determines that it is okay to proceed with the sterilization operation and operates the sterilization module to proceed with the sterilization operation. In the first paragraph, the HPE-based analysis unit
By analyzing the image collected by the above image collection unit, a human body (object) is detected, and at the same time, the pose of the detected object is detected using a previously learned pose detection algorithm, and the state-mode of the pose of the detected object is determined as 'temporary mode' or 'visit mode'.
The above controller
A mobile sterilization robot characterized in that it further includes a standby judgment unit that is executed when it is determined that the sterilization operation cannot be performed in the sterilization operation judgment unit, and 1) if there is an object whose status mode is 'visit mode' among the detected objects, it is finally determined not to perform the sterilization operation, but 2) if there is no object whose status mode is 'visit mode', it is determined to wait, and when a preset standby time (t) has elapsed, it restarts the image collection unit, the HPE-based analysis unit, and the sterilization operation judgment unit. In the second paragraph, the waiting or not judging unit
A state-mode input module that receives state-mode information of each object from the above HPE-based analysis unit;
A visit mode object search module that searches for the state-mode information of each object input through the above state-mode input module and searches whether there is an object among the objects whose state-mode is visit mode;
In the above-mentioned visit mode object search module, when an object having a visit mode status-mode exists among the objects, a movement decision module that outputs a movement command signal to the autonomous driving unit after the mobile sterilization robot decides not to wait in the corresponding area (S') but to move to the next area;
In the above visit mode object search module, when there is no object among the objects having the visit mode status-mode, a wait decision module that determines that the mobile sterilization robot will wait in the corresponding area (S') for a preset wait time (t);
An elapsed time measurement module that is executed when the above-mentioned waiting decision module decides to wait and measures the elapsed time (△t) using a timer;
An alarm display module that is executed when the above-mentioned standby decision module decides to wait and displays alarm information externally;
It includes a waiting time elapsed confirmation module that compares whether the elapsed time (△t) measured by the above elapsed time measurement module has elapsed the waiting time (t), and outputs a control signal to the image collection unit (34) when the elapsed time (△t) becomes the waiting time (t).
The above video collection unit
When a control signal is received from the above waiting time elapsed confirmation module, the previous process is repeated.
The above autonomous driving part
A mobile sterilization robot characterized in that, when a movement command signal is input from the above movement decision module, the next zone (S') is selected and the moving part is controlled to move to the next zone (S'). In the third paragraph, the controller
It further includes a memory in which matching information is stored between pose types belonging to the 'temporary mode' and pose types belonging to the 'visit mode'.
The above HPE-based analytics unit
An image analysis module that analyzes images collected by the above image collection unit;
A human body detection module that detects human bodies in an image by utilizing analysis data detected by the above image analysis module;
A PoseNet-based object joint detection module that uses the image of each object detected by the above-mentioned object detection module as input data for the PoseNet model and outputs joint points of each object;
An object-specific pose detection module that analyzes the joint points of each object detected by the above PoseNet-based object joint detection module and detects the pose of each object;
A mobile sterilization robot characterized by including a state-mode setting module that sets whether the state-mode of each object is a 'temporary mode' or a 'visit mode' by referring to the matching information stored in the above memory and the pose information of each object detected by the object-specific pose detection module. In the fourth paragraph, the sterilizing module
A plasma ion unit installed on the path of inhaled air and generating plasma ions;
UVA sterilization unit installed on the path of inhaled air and irradiating UVA;
A mobile sterilization robot characterized by including a UVC sterilization unit that irradiates UVC to a corresponding area (S').

Images