KR102541959B1 - Elevator control system and method for controlling elevator which robot and human board - Google Patents

Abstract

Based on the image taken inside the elevator car configured to allow the robot and a person to board, the space occupancy of the object(s) in the elevator car is calculated, and the robot is installed in the elevator car based on the calculated space occupancy level. An elevator control method for generating a signal for controlling the movement of an elevator car for boarding is provided.

Description

Method and elevator control system for controlling an elevator where robots and people board

The following description relates to a system and method for controlling an elevator for robot and human boarding.

Self-driving robots are robots that look around and detect obstacles on their own while finding the optimal path to their destination using wheels or legs.

A robot used to provide service in a building may need to board an elevator (elevator car) installed in the building to provide service on a specific floor of the building. However, when a robot gets on an elevator used by people, there are many cases where a robot and a person collide or interfere, or there is not enough space in the elevator for the robot to get on.

Therefore, according to the request of the robot or the robot control system, it is determined whether sufficient space is secured in the elevator for the robot to board, and the elevator control that can control the robot to board by determining the elevator suitable for the boarding of the robot. system is required.

Korean Patent Publication No. 10-2019-0002131 analyzes the number of users who want to use the elevator, learns and corrects it for each case, and even if the user enters the wrong number of elevator floors, it is possible to suggest the desired floor number based on big data. It discloses an artificial intelligent user-based elevator device and a control method thereof.

The information described above is for illustrative purposes only and may include material that does not form part of the prior art.

Based on an image taken inside an elevator car configured to allow a robot and a person to board, the degree of space occupancy (eg, occupancy rate or space occupancy information) of the object(s) in the elevator car is calculated, and the degree of space occupancy is calculated Provided is an elevator control method for generating a signal for controlling the movement of an elevator car in order to board a robot in the corresponding elevator car based on.

In order to calculate the degree of occupancy of the space of the object on board, a method of correcting the distortion caused by the camera and accurately calculating the degree of occupancy of the space by applying weights differently defined according to the position of the floor of the elevator car is provided.

According to a request from a robot or a robot control system, an elevator car having sufficient space for the robot to board is selected, and a method of controlling the selected elevator car for boarding of the robot is provided.

In one aspect, in an elevator control method performed by an elevator control system, based on an image of the inside of the elevator car photographed by a camera provided in an elevator car configured to allow a robot and a person to board, board in the elevator car Calculating a degree of occupancy of space for the elevator car by a running object, and based on the calculated degree of occupancy of space, controlling the elevator car to move to a position of the robot to board the robot in the elevator car. Including the step of generating a signal for, an elevator control method is provided.

The camera is a wide-angle camera installed at the center of the ceiling of the elevator car or at the center of the corner of the ceiling on the opposite side of the door of the elevator car, and may be configured to photograph the floor of the elevator car.

The step of calculating the degree of space occupancy may include detecting each of the objects in the elevator car from the image, the difference between the area of the floor of the elevator car and the area occupied by each of the objects, or the area of the floor The method may include calculating a ratio of an area occupied by each of the objects to an area and calculating the degree of space occupation based on the calculated difference or ratio.

The calculating of the difference or ratio may include projecting each of the detected objects on the floor and calculating the difference or ratio based on an area of the floor and an area occupied by each of the projected objects. can include

Calculating the difference or ratio comprises determining the position of each of the objects on the floor and applying to each of the objects a weight map defining a weight applied to each position on the floor. A step of calculating an area occupied by each of the objects may be included.

The weight map may be defined as having a larger weight value from the center of the floor toward a vertex of the floor, or as having a different weight for each zone for the floor divided into a plurality of zones. can

The calculating of the space occupancy may include a first space occupancy based on the calculated difference or ratio and a second space based on the number of objects in the elevator car for the maximum number of passengers in the elevator car. The method may include calculating a degree of occupancy and calculating the degree of occupancy of space based on the first degree of occupancy of space and the degree of occupancy of space based on the second degree of occupancy of space.

The degree of space occupation is calculated based on a sum of a value obtained by multiplying the first degree of space occupation by a first weight and a value obtained by multiplying the second degree of space occupation by a second weight, and the first weight is the second weight. It can be defined as a smaller value than

The step of detecting each of the objects may include detecting each of the objects by detecting the head of each of the objects in the image, and the head of each of the objects may be a person or a person from an image of a floor photographed in a vertical direction. It can be detected using a learning model trained to detect the robot's head.

The calculating of the difference or ratio may include segmenting each of the objects in the image to be distinguished from the floor, identifying a portion occupied by the segmented object among floor portions corresponding to the floor, and The method may further include calculating a ratio of the occupied portion to the floor as a ratio of an area occupied by the object to an area of the floor.

The elevator control method further includes receiving a call for an elevator car from the robot or a robot control system that controls the robot, and generating the signal based on the calculated degree of occupancy of space The method may include generating a signal for controlling an elevator car determined to be capable of being boarded by the robot to move to a position of the robot.

In the elevator control method, the elevator car moving to the location of the robot may be selected as having the calculated degree of occupancy of space less than a predetermined value among a plurality of elevator cars.

The call includes size information of the robot or required area information necessary for the robot to board in the elevator car, and the robot is placed in the elevator car based on the size information or the required area information and the calculated space occupation. Whether boarding is possible may be determined.

The elevator control method further includes receiving a call for an elevator car for boarding of the robots from at least one of a plurality of robots or a robot control system that controls the robots, and generating the signal The may include generating a signal for controlling an elevator car determined to be capable of being boarded by the robots based on the calculated degree of occupancy of space to be moved to a position of the robots for boarding the robots.

The degree of occupancy of the space may be calculated by analyzing the image based on information about the location where the camera is located, the maximum number of passengers in the elevator car, and information on the area of the floor of the elevator car.

In another aspect, an elevator control system for controlling an elevator car configured to allow a robot and a person to board, comprising at least one processor implemented to execute computer-readable instructions, wherein the at least one processor comprises the Based on an image of the interior of the elevator car taken by a camera provided in the elevator car, the degree of space occupancy of the elevator car by the object riding in the elevator car is calculated, and based on the calculated degree of occupancy of the space , An elevator control system is provided that generates a signal for controlling the elevator car to move to a position of the robot to board the robot in the elevator car.

In another aspect, in a method for calculating the degree of occupancy of an elevator car, performed by a computer system, the inside of the elevator car photographed by a camera installed in an elevator car configured to allow a robot and a person to board Based on the image, calculating the degree of space occupancy of the elevator car by the object riding in the elevator car, and the calculated space occupation degree to the position of the robot to board the robot in the elevator car A method for calculating the degree of occupancy of space is provided, comprising the step of transmitting to a system including a control panel for controlling the elevator car to move.

The degree of space occupancy of an object in the elevator car (ie, space occupancy or space occupancy information) can be calculated based on an image taken inside the elevator car without relying on a weight sensor or other sensors. That is, it is not possible to know the exact number of passengers in the elevator car with only the weight sensor, and even if a plurality of sensors are placed in the elevator car, it is not possible to accurately detect the degree of occupancy of space inside the elevator car. The degree of space occupancy inside the elevator car can be accurately calculated.

Accuracy in calculating the degree of space occupation may be increased by calculating the degree of space occupancy of an object in the elevator car by applying weights differently defined according to the location/region of the floor of the elevator car.

Floor space occupancy corresponding to the ratio of the floor area occupied by the object to the total space of the floor area corresponding to the first space occupancy degree and boarding for the maximum number of passengers in the elevator car corresponding to the second space occupancy degree By applying different weights to the ratio of the number of one object, it is possible to accurately calculate the degree of occupancy of the space of an object in an elevator car while minimizing an error due to image distortion.

According to the request of the robot or the robot control system, by selecting an elevator car that has enough space for the robot to board and controlling the selected elevator car for the robot to board, the efficiency of the elevator used by the robot and the person can be increased. there is.

1 shows an elevator control method performed by an elevator control system for controlling an elevator car configured to allow a robot and a person to board, according to an embodiment.
2 is a block diagram illustrating a robot providing service in a building, according to an exemplary embodiment.
3 is a block diagram illustrating an elevator control system for controlling an elevator car, according to one embodiment.
Figure 4 is a block diagram showing a robot control system for controlling the robot, according to an embodiment.
5 is a flowchart illustrating an elevator control method for controlling an elevator car configured to allow a robot and a person to board, according to an embodiment.
6 is a flowchart illustrating a method of calculating the degree of occupancy of a space of an object(s) on board based on an image inside an elevator car, according to an example.
7, 8, and 10 are flowcharts illustrating a method of calculating a difference between an area of an elevator car floor and an area of an object or a ratio of an area of an object to an area of an elevator car floor, according to an example.
9 is a flowchart illustrating a method of calculating the degree of space occupancy of object(s) in an elevator car according to an example.
11 shows a weight map used to calculate the degree of space occupancy of object(s) in an elevator car, according to an example.
12 and 13 illustrate a method of calculating the degree of space occupancy of an object(s) in an elevator car according to application of weights according to a weight map and application of ratio weights according to an example.
14 illustrates a result of identifying objects riding in an elevator car from an image of an inside of an elevator car and calculating a degree of occupancy of space by the objects according to an example.
15 shows a network configuration of an elevator control system for controlling an elevator car according to an example.
16 illustrates a method of processing an image taken inside an elevator car and controlling the elevator car in order to calculate the degree of occupancy of space, according to an example.
17 and 18 show a method of installing a camera for photographing the floor of an elevator car in an elevator car according to an example.
19 illustrates a result of segmenting an image of an inside of an elevator car according to an example.

Hereinafter, an embodiment will be described in detail with reference to the accompanying drawings.

1 shows an elevator control method performed by an elevator control system for controlling an elevator car configured to allow a robot and a person to board, according to an embodiment.

1, an elevator 20 (hereinafter referred to as an 'elevator car') installed in a building configured to allow humans and robots 100 to board, an elevator control system 130 for controlling the elevator car 20, A robot control system 140 for controlling the robot 100 is shown.

The robot 100 may be a service robot for providing service on at least one floor in a building.

The elevator car 20 is a device that moves between floors in a building, and in order to provide service, the robot 100 can ride and move the robot 100 between floors in the building. The elevator car 20 may be an elevator allowing both a person and a robot 100 to board. In other words, the elevator car 20 may be distinguished from a robot-only elevator in which only the robot 100 is allowed to board.

Although not shown, a plurality of elevator cars may be provided in a building where the elevator car 20 is provided, and the illustrated elevator car 20 may indicate any one of these plurality of elevator cars. Also, the robot 100 is one of a plurality of illustrated robots, and may represent a robot that wants to board the elevator car 20 by calling an elevator car to provide service. In the detailed description to be described later, the elevator car 20 and the robot 100 will also be described accordingly.

Elevator car 20 may be controlled by elevator control system 130 (or based on signals generated by elevator control system 130). For example, the elevator control system 130 may allow the robot 100 among a plurality of elevator cars to board ( For example, it is possible to select an elevator car 20 in which a space in which the robot 100 can board is secured, and the selected elevator car 20 is the position of the robot 100 (ie, the floor on which the robot 100 is located). can be moved to

When the elevator car 20 arrives at the location of the robot 100, the elevator control system 130 may control the door of the elevator car 20 so that the robot 100 gets on the elevator car 20.

Movement of the robot 100 and entry or exit of the robot 100 to or from the elevator car 20 may be controlled by the robot control system 140 .

A camera may be provided inside the elevator car 20, and the inside of the elevator car 20 may be photographed by the camera. This camera may be a wide-angle camera (a camera including a wide-angle lens) installed in the center of the ceiling of the elevator car 20 or in the center of the corner of the ceiling on the opposite side of the door of the elevator car 20. The installed camera may be configured to image (vertically or possibly vertically) the floor 50 of the elevator car 20 .

17 and 18 show a method of installing a camera for photographing the floor of an elevator car in an elevator car according to an example.

In FIG. 17, an example in which the camera is installed in the center of the corner of the ceiling on the opposite side of the door of the elevator car 20 is shown. In FIG. 18, an example in which the camera is installed in the center of the ceiling of the elevator car 20 is shown.

As shown in FIGS. 17 and 18 , the camera may be installed to vertically photograph the floor 50 , and the camera may have an angle of view capable of photographing the entire floor 50 . Unlike what is shown, the camera may be provided on the wall of the elevator car 20 .

In the embodiment, the object(s) (e.g., a robot, a person, or other The degree to which the object) occupies the internal space of the elevator car 20 (ie, the degree of space occupation or space occupation information) may be calculated. For example, the elevator control system 130 may receive an image of the inside of the elevator car 20 from a camera installed inside the elevator car 20, and analyze the received image to calculate the degree of occupancy of the space. The elevator control system 130 may control the elevator car 20 to move to the position of the robot 100 to board the robot 100 in the elevator car 20 based on the calculated degree of occupancy of space (or , can generate a signal for controlling the elevator car 20). When the elevator car 20 arrives at the location of the robot 100, the door of the elevator car 20 is opened, and the robot 100 can board the elevator car 20.

The operation of controlling the elevator car 20 by the elevator control system 130 and the operation of calculating the degree of occupancy of space may be performed when a request to call an elevator car is received from the robot 100 or the robot control system 140. there is. In other words, when a request to call an elevator car is received from the robot 100 or the robot control system 140, the elevator control system 130 can calculate the degree of occupancy of the space for each of the elevator cars, and the robot 100 gets on. It is possible to select the elevator car 20 having enough space to move to the position of the robot 100 and control it.

Alternatively, the elevator control system 130 is configured to calculate the degree of space occupancy by analyzing the image inside the elevator car 20 in real time or periodically regardless of a request from the robot 100 or the robot control system 140. It could be.

The “degree of space occupancy” is a parameter representing the degree to which an object in the elevator car 20 occupies the internal space of the elevator car 20, and may include, for example, space occupancy. For example, the degree of space occupancy is the difference between the area of the floor 50 of the elevator car 20 and the area occupied by the object on the elevator car 20 on the floor 50, or the area of the floor 50 It can be calculated based on the ratio of the area occupied by the object.

The elevator control system 130 can determine whether or not the robot 100 can board the elevator car 20 by calculating the degree of occupancy of the space for the elevator car 20, and therefore, the boarding of the robot 100 You can select and control the suitable elevator car.

Therefore, according to the embodiment, without determining whether the elevator car 20 is full by the weight sensor included in the elevator car 20, by analyzing the image of the inside of the elevator car 20, the elevator car 20 The degree of space occupancy can be calculated. In addition, in the embodiment, it is possible to eliminate the need to install various sensors inside the elevator car 20 to calculate the degree of space occupancy of the elevator car 20 .

Meanwhile, the method of calculating the degree of occupancy of space and the method of controlling the elevator car 20 according to the embodiment are additionally (or auxiliaryly) to the method of calculating the degree of occupancy of space using a weight sensor (and other sensors). ) may be used.

A method of calculating the degree of space occupancy of the elevator car 20 and a specific method of controlling the elevator car 20 will be described in more detail with reference to FIGS. 2 to 16 and 19 to be described later.

2 is a block diagram illustrating a robot providing service in a building, according to an exemplary embodiment.

The robot 100 may be a service robot used to provide services within a building. Robot 100 may be configured to provide service on at least one floor of a building. Also, when there are a plurality of robots 100, each of the plurality of robots may be configured to provide services on at least one floor. In other words, depending on the type/frequency of service and/or the shape/structure of the building (floor), the robot 100 may be configured to provide service on one or more floors, and a plurality of robots may operate on one floor. It may also be configured to provide a service. The robot 100 may provide services to users in a building at a predetermined location (specific floor) of a building through autonomous driving.

The service provided by the robot 100 may include, for example, at least one of a parcel delivery service, a beverage (coffee, etc.) delivery service according to an order, a cleaning service, and other information/content providing services.

At least part of the movement of the robot 100, provision of services, and calls to elevator cars may be performed through control by the robot control system 140. For example, in response to a call by the robot control system 140, the elevator control system 130 may call an appropriate elevator car 20 to the floor where the robot 100 is located.

The robot 100 may be a physical device and, as shown, may include a control unit 104, a drive unit 108, a sensor unit 106, and a communication unit 102.

The control unit 104 may be a physical processor built into the robot 100, and although not shown, a route planning processing module, a mapping processing module, a drive control module, a localization processing module, a data processing module, and a service processing module can include At this time, the path planning processing module, the mapping processing module, and the localization processing module according to the embodiment in order to enable indoor autonomous driving of the robot 100 even when communication with the robot control system 140 is not performed Optionally, it may be included in the control unit 104.

The communication unit 102 may be a component for the robot 100 to communicate with other devices (such as other robots or robot control systems 140). In other words, the communication unit 102 is a hardware module or network device such as an antenna, data bus, network interface card, network interface chip, and networking interface port of the robot 100 that transmits/receives data and/or information to other devices. It can be a software module such as a driver or networking program.

The driving unit 108 controls the movement of the robot 100 and may include equipment for enabling the movement.

The sensor unit 106 may be a component for collecting data required for autonomous driving of the robot 100 and service provision. The sensor unit 106 may not include expensive sensing equipment, and may include only sensors such as a low-cost ultrasonic sensor and/or a low-cost camera.

For example, the data processing module of the control unit 104 may transmit sensing data including output values of the sensors of the sensor unit 106 to the robot control system 140 through the communication unit 102 . The robot control system 140 may transmit path data generated using an indoor map in a building to the robot 100 . Route data may be transmitted to the data processing module through the communication unit 102 . The data processing module may directly transmit the route data to the driving control module, and the driving control module may control the indoor autonomous driving of the robot 100 by controlling the driving unit 108 according to the route data.

When the robot 100 and the robot control system 140 cannot communicate, the data processing module transmits the sensing data to the localization processing module, and generates path data through the path planning processing module and the mapping processing module to generate the robot ( 100) may directly process indoor autonomous driving.

The robot 100 may be different from a mapping robot used to create an indoor map in a building. At this time, since the robot 100 does not include expensive sensing equipment, it can process indoor autonomous driving using an output value of a sensor such as a low-cost ultrasonic sensor and/or a low-cost camera. On the other hand, if the robot 100 has previously processed indoor autonomous driving through communication with the robot control system 140, mapping data including the path data previously received from the robot control system 140 may be further transmitted. By using low-cost sensors, more accurate indoor autonomous driving may be possible.

The service processing module may receive commands received through the robot control system 140 through the communication unit 102 or through the communication unit 102 and the data processing module. The driving unit 108 may further include equipment related to services provided by the robot 100 as well as equipment for moving the robot 100 . For example, in order to perform a food/delivery delivery service, the driving unit 108 of the robot 100 is configured to load food/delivery or deliver food/delivery to a user (eg, a robot arm). can include In addition, the robot 100 may further include a speaker and/or a display for providing information/content. The service processing module may transmit a driving command for a service to be provided to the driving control module, and the driving control module may control the configuration included in the robot 100 or the driving unit 108 according to the driving command to provide the service. can make it

The robot 100 can detect the called elevator car 20 through control by the robot control system 140 and can board the elevator car 20 . When the elevator car 20 arrives at a floor for the robot 100 to provide service, the robot 100 may get off the elevator car 20 and provide service on the corresponding floor.

When specific control is performed by the robot control system 140, the robot 100 may correspond to a brainless robot in that it only provides sensing data for control of the robot 100 to the robot control system 140. there is.

The structure of the elevator control system 130 and the robot control system 140 will be described in more detail with reference to FIGS. 3 and 4 to be described later.

Descriptions of the technical features described above with reference to FIGS. 1, 17, and 18 may be applied to FIG. 2 as they are, and thus duplicate descriptions are omitted.

3 is a block diagram illustrating an elevator control system for controlling an elevator car, according to one embodiment.

The elevator control system 130 controls the movement of the elevator car 20 and calls for the elevator car 20 moving (eg, going up and down) in the building (or a signal for controlling the movement of the elevator car 20). It may be a device that generates). Elevator control system 130 may include at least one computing device and may be implemented as a computer system located inside or outside a building.

The elevator control system 130 may be different from a control panel that directly controls the elevator car 20 . The elevator control system 130 may transmit signals required to control the elevator car 20 to the control panel. Alternatively, the elevator control system 130 may be configured to include a control panel.

The elevator control system 130 may be provided inside the AI camera or implemented as a computer system capable of communicating with the camera when the camera that photographs the inside of the elevator car 20 is an AI camera. For example, it may be configured to include an FPGA SoC included in the elevator control system 130 and a gateway board.

As shown, the elevator control system 130 may include a memory 330, a processor 320, a communication unit 310, and an input/output interface 340.

The memory 330 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, the ROM and the non-perishable mass storage device may be separated from the memory 330 and included as a separate permanent storage device. Also, an operating system and at least one program code may be stored in the memory 330 . These software components may be loaded from a computer-readable recording medium separate from the memory 330 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, software components may be loaded into the memory 330 through the communication unit 310 rather than a computer-readable recording medium.

The processor 320 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processor 320 by the memory 330 or the communication unit 310 . For example, processor 320 may be configured to execute instructions received according to program code loaded into memory 330 .

The processor 320 may include software and/or hardware module(s) as components not shown. An elevator control method to be described later may be performed through the components of the processor 320 . That is, the processor 320 may calculate the degree of space occupancy of the elevator car 20 by analyzing an image of the inside of the elevator car 20, and for the robot 100 calling the elevator car, the robot 100 ) can determine the elevator car 20 having enough space to board and move it to the position of the robot 100. Calculation of the degree of space occupancy and control of the elevator car 20 may be performed by separate processors or modules.

The communication unit 310 may be a component for the elevator control system 130 to communicate with other devices (such as the elevator car 20 or the robot control system 140). In other words, the communication unit 310 is a hardware module such as an antenna, a data bus, a network interface card, a network interface chip, and a networking interface port of the elevator control system 130 that transmits/receives data and/or information to other devices, or It may be a software module such as a network device driver or networking program.

The input/output interface 340 may be a means for interfacing with an input device such as a keyboard or mouse and an output device such as a display or speaker.

Also, in other embodiments the elevator control system 130 may include more components than those shown.

A more specific method for the elevator control system 130 to calculate the degree of space occupancy of the elevator car 20 and to control the elevator car 20 will be described in more detail with reference to FIGS. 5 to 16 and 19 to be described later. do.

Descriptions of the technical features described above with reference to FIGS. 1, 2, 17, and 18 may be applied to FIG. 3 as they are, and thus duplicate descriptions are omitted.

Figure 4 is a block diagram showing a robot control system for controlling the robot, according to an embodiment.

The robot control system 140 may be a device that controls movement of the robot 100 and provision of services by the robot 100 in a building. The robot control system 140 may control the movement of each of the plurality of robots and the provision of services of each of the robots. The robot control system 140 may call the elevator car 20 to move the robot 100 to the floor where the robot 100 will provide service through communication with the elevator control system 130 . The robot control system 140 controls the robot 100 so that the robot 100 can recognize and board the called elevator car 20, and the robot 100 to get off the elevator car 20 at the floor to provide service ( 100) can be controlled.

The robot control system 140 may include at least one computing device and may be implemented as a server located inside or outside a building. The robot control system 140 may be implemented as a cloud server (system).

As shown, the robot control system 140 may include a memory 430, a processor 420, a communication unit 410, and an input/output interface 440. For the general description of the components (410 to 430) of the robot control system 140, the description of the general technical characteristics of the components (310 to 340) of the elevator control system 130 described above can be applied as it is, so redundant omit explanation.

On the other hand, depending on the implementation method, the elevator control system 130 and the robot control system 140 may be implemented as one single system 125. In other words, the elevator control system 130 and the robot control system 140 may be implemented as a single server or computing device that resides within or outside a building.

Descriptions of technical features described above with reference to FIGS. 1 to 3, 17, and 18 may be applied to FIG. 4 as they are, and thus duplicate descriptions are omitted.

In the detailed description to be described later, the operation performed by the components of the elevator control system 130 or the robot control system 140 is an operation performed by the elevator control system 130 or the robot control system 140 for convenience of description. can be explained

In the detailed description to be described later, "calculation" of the degree of space occupation (space occupation information) refers to performing an operation for calculating the degree of space occupation, generating the degree of (computed) space occupation, and determining the degree of (computed) space occupation. It can be used in an inclusive sense.

5 is a flowchart illustrating an elevator control method for controlling an elevator car configured to allow a robot and a person to board, according to an embodiment.

In step 520, the elevator control system 130, based on the image of the inside of the elevator car 20 captured by the camera provided in the elevator car 20 configured to allow the robot 100 and a person to ride, the elevator car In (20), it is possible to calculate the degree of space occupancy (space occupancy information) of the elevator car by the object being boarded.

Each of the objects aboard the elevator car 20 may be, for example, a robot, a person or other object. The elevator control system 130 may calculate the degree of space occupancy as an index representing the degree of space occupied by the object with respect to the inner space of the elevator car 20 . The degree of space occupancy may also be referred to as space occupancy information.

Elevator control system 130, for example, the difference between the area of the floor 50 of the elevator car 20 and the area occupied by the object on the elevator car 20 on the floor 50, or the floor 50 The degree of occupancy of the space may be calculated based on the ratio of the area occupied by the object to the area of .

The image to be analyzed for calculating the degree of space occupancy may be an image obtained by photographing the floor 50 of the elevator car 20 (vertically or vertically as much as possible) and photographing the entire area of the floor 50. there is.

The degree of space occupancy is determined by location information (in the elevator car 20) where the camera is provided (position of the ceiling where the camera is provided and height from the floor, etc.), the maximum number of passengers for the elevator car 20, and the number of people on the elevator car 20. Based on the area information of the floor 50, it can be calculated by analyzing an image of the inside of the elevator car 20. In addition, in order to calculate the degree of space occupancy, parameters of the camera (view angle information of the camera and parameters related to photographing, etc.) may be further used.

The elevator control system 130 may recognize a portion of the floor 50 from the image of the inside of the elevator car 20, and may calculate an area (initialization area) of the floor 50. The elevator control system 130 may calculate the degree of occupancy of the space by recognizing an object in the image and calculating an area occupied by the recognized object. A more specific method for calculating the degree of space occupancy will be described later.

In step 530, the elevator control system 130 uses the robot 100 to board the robot 100 in the elevator car 20, based on the degree of space occupation calculated in step 520 (space occupation information). It is possible to control the elevator car 20 to move to the position of ). Alternatively, the elevator control system 130 may generate a signal for controlling the elevator car 20 to move to the position of the robot 100 in order to board the robot 100 in the elevator car 20 . The elevator control system 130 determines an elevator car 20 having enough space for the robot 100 to board, and assigns the elevator car 20 to the position of the robot 100 (ie, the robot 100 is waiting). You can move it to the location/floor where you are on or waiting for it).

As described above, the control of the elevator car 20 in step 530 is controlled by the elevator control system 130 from the robot 100 or the robot control system 140 that controls the robot 100 to the elevator car. It may be based on receiving a call.

In step 510, the elevator control system 130 may receive a call for an elevator car from the robot 100 or the robot control system 140 that controls the robot 100. Unlike shown, step 510 may be performed after step 520 .

In response to a call from the robot 100 or the robot control system 140 that controls the robot 100, the elevator control system 130 allows the robot 100 to board based on the degree of occupancy of space calculated in step 520. The elevator car 20 determined to be capable can be controlled to move to the position of the robot 100 (or the elevator control system 130 can generate a signal to control the elevator car 20). The elevator car 20 may be one selected from among a plurality of elevator cars controlled by the elevator control system 130 after it is determined that the robot 100 can board.

For example, the elevator control system 130 determines that the elevator car 20 can be boarded by the robot 100 when the calculated space occupancy degree for the elevator car 20 is less than a predetermined value, and the elevator car 20 ) can be controlled to move to the position of the robot 100 (or, the elevator control system 130 can generate a signal for controlling the elevator car 20). At this time, the elevator car 20 moving to the position of the robot 100 may be one selected from among a plurality of elevator cars controlled by the elevator control system 130 as having a calculated degree of occupancy of space less than a predetermined value.

In the case where robots of various sizes/types are used, the elevator control system 130 may apply different criteria according to the type or size of the robot 100 that has called the elevator car to determine an elevator car that the robot 100 can board. there is.

A call from the robot 100 or the robot control system 140 may include size information of the robot 100 or required area information required for the robot to ride in an elevator car. Alternatively, such a call may include information about the type of robot 100 or information about a service provided by the robot 100 .

The elevator control system 130 may determine whether the robot can board the elevator car 20 based on the size information or required area information of the robot 100 and the calculated space occupancy degree for the elevator car 20. .

For example, the elevator control system 130 calculates the degree of space occupancy of the elevator car 20 according to the required area information of the robot 100 (or the required area information calculated from the size information of the robot 100). If it can accommodate more than this amount, it may be determined that the robot 100 can board the elevator car 20 .

In other words, the elevator control system 130 determines that the elevator car 20 can be boarded by the robot 100 when the calculated space occupancy degree for the elevator car 20 is less than a predetermined value, and The elevator car 20 may be moved to the position of the robot 20 for boarding. At this time, the predetermined value may be set differently according to size information or required area information included in the call.

The required area information may be included in the call and provided, but may also be generated by the elevator control system 130 . For example, the elevator control system 130 requests the robot 100 based on information on the size of the robot 100 included in the call, information on the type of the robot 100, or information on the service provided by the robot 100. Requested area information corresponding to the area may be generated. For example, if the robot 100 performs a task with a relatively high risk or a service provided by the robot 100 is a relatively high risk (eg, transportation of hot food), the robot 100 can be determined to require a larger area.

Meanwhile, the elevator control system 130 may receive a call for an elevator car for boarding of a plurality of robots. The call at this time may be received from at least one of the plurality of robots (eg, a representative robot selected from among the robots) or the robot control system 140 that controls the robots.

The call of the elevator control system 130 may further include information on the quantity of robots. The elevator control system 130 further increases the area of the space occupancy calculated for the elevator car 20 by the area according to the required area information of the robots determined according to the quantity information (and the size/type/service information of the robots). If it can be accommodated, it can be determined that the robots can board the elevator car 20 . In one example, the required area of the robots may be calculated based on the number of robots included in the call and the size (or area required) of each robot.

Accordingly, the elevator control system 130 determines whether or not robots can board the elevator car 20 based on the calculated degree of occupancy of the space of the elevator car 20, and determines whether the robots can board the elevator car 20 ) can be moved to the position of the robots. When the robots are located on different floors, the elevator car 20 may be controlled to sequentially move the floor on which each robot is located.

As described above, according to the embodiment, the robot 100 or the elevator car in which the robots can ride can be determined and controlled based on the degree of space occupation calculated based on the image taken inside the elevator car, so that people and Interference between a human and a robot can be minimized in using an elevator car that is used together with a robot.

A more specific method of calculating the degree of occupancy of space based on an image of the inside of the elevator car 20 will be described later.

Description of the technical features described above with reference to FIGS. 1 to 4, 17 and 18 may be applied to FIG. 5 as it is, and thus duplicate descriptions are omitted.

6 is a flowchart illustrating a method of calculating the degree of occupancy of a space of an object(s) on board based on an image inside an elevator car, according to an example.

Referring to FIG. 6, the method of calculating the degree of space occupancy in step 520 described above will be described in more detail.

In step 610, the elevator control system 130 may detect each of the objects inside the elevator car 20 from the image taken inside the elevator car 20.

The elevator control system 130 may detect an object inside the image by analyzing an image of the inside of the elevator car 20 using an artificial intelligence-based model. The image analyzed using the artificial intelligence-based model may be an image obtained by preprocessing (eg, distortion correction, cropping, scaling, rotation, etc.) of an image taken inside the elevator car 20 .

The elevator control system 130 may detect each of the objects by detecting the head of each object (included in the image) in the image. The head part is an upper part of the object and may be a specific part of the object identified when the object is viewed from above. For example, the elevator control system 130, as an artificial intelligence-based model, is an image of a learning model trained to detect the head/top of a human or robot from an image of the floor 50 of the elevator car 20 taken in a vertical direction. It is possible to detect the head/top part of a person or robot from The learning model may be trained using training images including object(s), which are vertically photographed on the floor of an elevator car or a floor in a space.

By detecting the object by detecting the head, it is possible to minimize detection errors due to overlap in object detection in the image.

The learning model is an artificial neural network-based model, and may be a CNN-based model or a deep learning (DNN, ANN, RNN, etc.)-based model. The learning model may be further trained (updated) based on images of the interior of the elevator car 20 used to calculate the degree of occupancy of space.

The learning model may be trained to exclude an object reflected from a mirror or a wall provided in the elevator car 20 or a part corresponding to the shadow of the object.

In addition, the learning model can be further trained to detect a person's shape in order to detect a person more accurately.

Also, the learning model can be updated by additionally learning an object that frequently gets on the elevator car 20 .

As for the method of training the learning model in the embodiment and the method of identifying/detecting objects using the learning model, any method of identifying/detecting objects by any artificial intelligence-based model can be applied. omit

In step 620, the elevator control system 130 determines the difference (first value) between the area of the floor 50 of the elevator car 20 and the area occupied by each of the detected objects, or the area of the floor 50. A ratio (a second value) of an area occupied by each of the detected objects (or an area occupied by the entire object) may be calculated.

The elevator control system 130 may recognize a portion of the floor 50 from the image of the inside of the elevator car 20, and may calculate an area (initialization area) of the floor 50. The calculated area of the floor 50 may be a relative value. The elevator control system 130 may calculate the first value or the second value by calculating an area occupied by the object detected in the image with respect to the floor 50 .

For example, the first value may correspond to a value obtained by subtracting an area occupied by each of the detected objects from the area of the floor 50 . The second value may correspond to a value obtained by dividing an area of the floor 50 by an area occupied by each of the detected objects on the floor 50 (or an area occupied by the entire object on the floor 50 ). Alternatively, the first value may be a value obtained by subtracting the second value from 1.

At step 630, the elevator control system 130 may calculate the amount of space occupancy (space occupancy information) for the elevator car 20 based on the difference or ratio calculated at step 620. The calculated space occupancy degree (information) is the final space occupancy degree for the elevator car 20, and the floor occupancy degree based on the first value or the second value calculated in step 620 (the first space occupation degree to be described later) and , It may be calculated based on the occupancy level (the second space occupancy level to be described later) based on the maximum number of passengers and the number of objects on the elevator car 20 .

A more specific method of calculating the first value or the second value of step 620 and a specific method of calculating the final space occupancy will be described later.

Descriptions of the technical features described above with reference to FIGS. 1 to 5, 17, and 18 may be applied to FIG. 6 as they are, so duplicate descriptions are omitted.

7, 8, and 10 show a difference between the area of the floor of an elevator car and the area of an object (the above-mentioned first value) or the ratio of the area of an object to the area of the floor of an elevator car (the above-mentioned first value) according to an example. It is a flowchart showing a method of calculating the second value).

In this regard, FIG. 7 describes a method of calculating the degree to which the object occupies the floor 50 by projecting the detected object onto the floor 50 .

In step 710, the elevator control system 130 may project each of the detected objects onto the floor 50 recognized in the image. At this time, the floor 50 is a part recognized according to the image pre-processing and may correspond to a rectangular plane. A part corresponding to the object detected in the image may be projected onto the floor 50, and the elevator control system 130 distinguishes between a part on which the object is projected and a part of the floor 50 on which the object is not projected and recognizes it. can

In step 720, the elevator control system 130 determines a difference (eg, relative) area of the floor 50 and a difference (eg, relative) area occupied by each of the objects projected on the floor 50. value) or ratio (second value) can be calculated.

As the elevator control system 130 distinguishes and recognizes a portion corresponding to an object projected on a plane corresponding to the floor 50, the area of the portion where the object is projected and the area of the floor 50 are compared to the first value or the second value may be calculated.

Meanwhile, depending on the position of the object on the floor 50 (or the direction in which the object is located), distortion of an area actually occupied by the object may occur. In order to correct the distortion generated with respect to the area occupied by the object, a weight applied differently according to the position of the floor 50 may be applied to the object.

8 is a method of calculating an area occupied by an object (corrected area) for calculating the first value or the second value by applying a weight applied differently according to the location of the floor 50 to the object. explain

In step 810, the elevator control system 130 may determine a position on each floor 50 of the object detected in the image. For example, the elevator control system 130 may determine positions corresponding to points (eg, pixels) constituting each object. The object may be projected onto the floor 50 as described above with reference to FIG. 7 , and the location of the object may be determined as a location on a plane corresponding to the floor 50 (ie, the location of the projected object). there is.

In step 820, the elevator control system 130 applies a weight map defining a weight applied to each position on the floor 50 to each of the detected objects so that the area occupied by each of the objects (correction) area) can be calculated. For example, the elevator control system 130 applies a weight map defining weights applied to each position on a plane corresponding to the floor 50 to each position of an object projected on the floor 50, thereby corresponding to the corresponding position. The area occupied by each of the objects (corrected area) can be calculated. The weight may be, for example, a value of 0 to 1, and application of the weight may be an operation of multiplying the weight by a value of a position of a corresponding object.

The weight map may be defined as having a larger weight value from the center of the floor 50 (a plane corresponding to the floor 50) toward a vertex of the floor 50. Alternatively, the weight map may be defined as having different weights for each zone with respect to the floor 50 divided into a plurality of zones. For example, a plane corresponding to the floor 50 may be divided into 9 zones (ie, 3*3 zones), and a weight map may be defined such that appropriate weights are allocated according to characteristics of each zone.

In this regard, FIG. 11 shows a weight map used to calculate the degree of space occupancy of object(s) on the elevator car 20 according to an example. The illustrated weight map may be used to calculate the first value and the second value. The illustrated weight map is defined so that a larger value of weight is allocated from the center of the bottom 50, which is a dark portion, toward the vertex of the bottom 50, which is a bright portion. The weight map may be differently defined according to a location where a camera is provided in the elevator car 20, a parameter of the camera, and a method in which an object is distorted in a captured image.

By correcting the area occupied by the object on the floor 50 using such a weight map, an error in which the area occupied by the object on the floor 50 is erroneously recognized according to image distortion can be minimized. Accordingly, the accuracy of the calculated final space occupancy degree for the elevator car 20 can be increased.

In this regard, FIGS. 12 and 13 show a method of calculating the degree of space occupancy of an object(s) in an elevator car according to application of weights according to a weight map and application of ratio weights according to an example. The application of ratio weights will be described later with reference to FIG. 9 , and here, the application of weights according to weight maps will be described.

The first figure of FIG. 12 shows the ratio of the area occupied by the object to the floor 50 according to the position of the object on the floor 50 when the weight by the weight map is not applied.

The second figure of FIG. 12 shows the ratio of the area occupied by the object to the floor 50 according to the position of the object on the floor 50 when the weight by the weight map is applied.

In FIG. 12 , a ratio of an area occupied by an object to the floor 50 when the floor 50 is divided into 3×3 zones and an object is located in each zone is shown.

FIG. 13 shows an image of the inside of the elevator car 20 when the floor 50 is divided into 3*3 zones and an object is located in each zone.

Distortion may occur in the ratio of the object occupying the floor 50 due to the location where the camera is installed (the center of the corner of the ceiling opposite to the door or the center of the ceiling) and the characteristics of the camera's wide-angle lens.

In FIG. 13, it is assumed that the camera is provided in the center of the corner of the ceiling on the opposite side of the door. In this case, distortion may increase as the position of the object moves away from the center of the floor 50 (x-axis direction). For example, when an object is located on the left or right side, distortion may appear in which the area occupied by the floor 50 is reduced because the object appears tilted in the image. Also, distortion may increase as the object is positioned closer to the door (y-axis direction). For example, distortion in which the area occupied by the floor 50 becomes wider due to distortion may occur in the area where the object is 2/3 from the door, and at 1/3 point from the door, the object moves toward the door and occupies the floor 50. Distortion may occur in which the area becomes smaller.

As shown in the first picture of FIG. 12 , it can be confirmed that the ratio occupied by the object with respect to the floor 50 is distorted when the weight by the weight map is not applied. That is, it can be confirmed that the deviation of the ratio occupied by the object with respect to the floor 50 according to the position of the object increases.

In contrast, as shown in the second figure of FIG. 12 , when weights are applied using the weight map, it can be confirmed that the deviation of the ratio occupied by the object with respect to the floor 50 according to the position of the object is further reduced.

Therefore, by correcting the area occupied by the object on the floor 50 using the weight map shown in FIG. 11 , an error in which the area occupied by the object on the floor 50 is erroneously recognized according to image distortion can be minimized. there is. Accordingly, the accuracy of the calculated final space occupancy degree for the elevator car 20 can be increased.

Meanwhile, referring to FIG. 10 , a method of using a segmented image as a method of calculating the first value and the second value will be described.

In step 1010, the elevator control system 130 may segment each object detected in the image inside the elevator car 20 to be distinguished from the floor. Segmentation may be to visually distinguish each of the objects and the floor from each other. For example, each of the objects may be overlaid on the image with a different color (see FIG. 12).

19 shows a result of segmenting an image of the inside of an elevator car 20 according to an example. As shown, the floor 1910, the head 1920 of the object, and other parts (such as the body) 1930 of the object may be displayed to be visually distinguished from each other. Segmentation of an image may be drawing each object in the image.

The segmented image 1900 may be provided so that an administrator of the elevator control system 130 can check (eg, in a test environment of the elevator control system 130).

In step 1020, the elevator control system 130 may identify occupied portions 1920 and 1039 occupied by the segmented object among the floor portion 1910 corresponding to the floor 50.

At step 1030, the elevator control system 130 may calculate ratios 1920 and 1930 of the occupied portion to the floor portion 1910 as the ratio of the area occupied by the object to the area of the floor 50. That is, the elevator control system 130 may calculate the first value or the second value from the segmented image.

The technical details of the method for calculating the degree of space occupancy from an image described above with reference to FIGS. 5 to 8 may be similarly applied to a segmented image.

The description of the technical features described above with reference to FIGS. 1 to 6, 17, and 18 may be applied as it is to FIGS. 7, 8, 10 to 13, and 19, so duplicate descriptions are omitted. .

9 is a flowchart illustrating a method of calculating the degree of space occupancy of object(s) in an elevator car according to an example.

Referring to FIG. 9, as the degree of occupancy of space (space occupation information) for the elevator car 20, the degree of occupancy of the floor based on the first value or the second value calculated in step 620 (degree of occupancy of the first space); A method of calculating the final space occupancy degree based on the maximum number of passengers and the number of objects boarded in the elevator car 20 (a second space occupation degree to be described later) will be described.

In step 910, the elevator control system 130 determines the first space occupancy degree based on the difference or ratio calculated in step 620 described above and the elevator car for the maximum number of passengers for the elevator car 20 ( 20) It is possible to calculate the second space occupancy level based on the number of objects on board.

The first space occupancy degree represents the occupancy degree of an object boarding the elevator car 20 occupying the floor 50, and may be, for example, floor space occupancy. The first space occupation degree may correspond to the first value or the second value described above with reference to FIGS. 7 , 8 , and 10 . The floor space occupancy may be a value expressed by the following equation.

[Equation 1]

Floor space occupancy = (occupied part of the floor surface) / (total floor space)

The occupied portion of the floor surface may refer to a recognized portion of an object (person and/or object) among portions of the floor 50 (eg, segmented portions of the floor 50) in the image. The entire space of the floor may mean an entire part corresponding to the floor 50 in the image.

The second space occupancy degree is a ratio of the number of currently boarded objects to the maximum number of passengers on the elevator car 20, and may be, for example, occupancy rate. The occupancy rate may be a value expressed by the following equation.

[Equation 2]

occupancy rate = number of boarding objects/maximum number of boarding people

final share.

The degree of occupancy of space (final degree of occupancy of space) for the elevator car 20 may be calculated by comprehensively considering the first degree of occupancy of space and the degree of second space occupation.

In step 920, the elevator control system 130 may calculate space occupancy information (final space occupancy level) based on the first space occupancy level and the second space occupancy level. For example, the elevator control system 130 may calculate the final space occupation based on the sum of the value obtained by multiplying the first space occupation by the first weight and the second space occupation by the second weight. In this case, the first weight may be defined as a smaller value than the second weight. The first weight and the second weight may be appropriately adjusted by a manager or automatically adjusted by the elevator control system 130 according to operating conditions of elevator cars and robots.

That is, in calculating the final space occupancy of the embodiment, the first space occupancy (floor space occupancy) may be considered smaller than the second space occupancy (person occupancy).

This is because the extent to which the object occupies the floor 50 is not accurately determined by the portion covered in the image. For example, when an object is located in the center of the floor 50, a problem in that the area occupied by the floor 50 becomes excessively large may occur. In addition, distortion may occur in an image in which an object is tilted. In addition, according to the angle of the camera, various environmental factors, and the motion of the object, a change in the occupancy rate of the floor space may occur, resulting in distortion.

Distortion can be reduced by using the above-mentioned weight map, but for more accurate calculation of the final space occupancy degree, the proportion of the first space occupancy degree (floor space occupancy) is converted into a ratio weight (the above-mentioned first weight and second weight). can be further reduced by using

In other words, in order to reduce the maximum/minimum error range, the percentage weight of floor space occupancy may be further reduced compared to the percentage weight of occupancy of personnel.

The third figure of FIG. 12 shows the ratio of the area occupied by the object to the floor 50 according to the position of the object on the floor 50 when more ratio weights are applied compared to the case of the second figure of FIG. 12 . .

Comparing the above-mentioned second picture of FIG. 12 with the third picture of FIG. 12, the case where the ratio weight is applied has a higher deviation of the ratio occupied by the object with respect to the floor 50 according to the position of the object than the case where the ratio weight is not applied. You can see that it decreases.

The final space occupancy degree to which the ratio weight is applied can be calculated as in Equation 3 below.

[Equation 3]

Final space occupancy (final occupancy) = (person occupancy) x 0.8 + floor space occupancy x 0.2

Meanwhile, the corrected final space occupancy degree obtained by further adjusting the ratio weights (the above-mentioned first weight and second weight) may be calculated as in Equation 4 below.

[Equation 4]

Corrected final space occupancy rate (final occupancy) = (person occupancy) x 0.9 + floor space occupancy x 0.1

By calculating the corrected final space occupancy degree as in Equation 4, the effect of distortion on the occupancy rate of floor space can be minimized. In this case, the first weight may be 0.1 and the second weight may be 0.9.

The description of the technical features described above with reference to FIGS. 1 to 8, 10 to 13, and 17 to 19 may be applied to FIG. 9 as it is, and thus duplicate descriptions are omitted.

14 illustrates a result of identifying objects riding in an elevator car from an image of an inside of an elevator car and calculating a degree of occupancy of space by the objects according to an example.

In FIG. 14 , objects are detected according to the analysis of the image, and a result of calculating the degree of space occupancy of the objects in the elevator car 20 is shown.

As shown, two people (I 2 ) and one robot (R 1 ) were detected as objects. The degree of space occupancy by these objects was calculated to be 14% (A 14%).

Humans and robots as objects can be recognized separately from each other. For example, a learning model for object detection may be trained to detect a human head and a robot top as different types of objects.

The learning model may be additionally trained to detect various types/sizes of robots.

On the other hand, in the embodiment, it may be assumed that the robot occupies the same area as the human, and therefore, it may be treated the same in calculating the final space occupancy degree. In other words, in calculating the occupancy rate, one robot can be treated the same as one person. Depending on the embodiment, robots and humans may be treated differently in calculating the occupancy rate.

Since the description of the technical features described above with reference to FIGS. 1 to 13 and 17 to 19 may be applied to FIG. 14 as it is, duplicate descriptions will be omitted.

15 shows a network configuration of an elevator control system for controlling an elevator car according to an example.

The above-described elevator control system 130 may be configured to include the illustrated gateway. The elevator control system 130 may control the elevator cars through CAN communication. Deep learning processing on the image received through the camera may be performed in the gateway.

A gateway can communicate with an AP through a LAN, and an AP can be connected to an external network through a LAN. According to the image processing by the gateway, the result values of the processing (e.g., signals for elevator car control, the number of boarding objects, space occupancy, etc.) are transferred to the CTC and the control panel through CAN to control the elevator car. can be used for Video can be transmitted to an external network through a LAN.

Descriptions of the technical features described above with reference to FIGS. 1 to 14 and 17 to 19 may be applied to FIG. 15 as they are, so duplicate descriptions are omitted.

16 illustrates a method of processing an image taken inside an elevator car and controlling the elevator car in order to calculate the degree of occupancy of space, according to an example.

In the embodiment, a camera including a wide-angle lens may be installed inside the elevator car to photograph the floor in a vertical direction, and the image collected from the camera is pre-processed-deep-learning-post-processed so that the elevator car is controlled according to the result The robot may be able to board the corresponding elevator car.

As shown, an image captured by a camera may be transmitted (eg, to an external network or an entity that views/manages the original image) using RTP (Realtime Transport Protocol) as an original image after distortion correction and JPEG compression. Meanwhile, in order to analyze the degree of space occupancy, an image captured by a camera may be cropped/scaled (pre-processed) and then analyzed through an artificial neural network. After the post-processing operation, the result values of the processing (eg, signals for elevator car control, number of boarding objects, space occupancy, etc.) may be transmitted for control of the elevator car through CAN. On the other hand, the image (deep learning image) after the post-processing is completed is to be transmitted to a subject (eg, the manager of the elevator control system 130) who controls the elevator car and remotely monitors the degree of space occupancy of the elevator car using RTP. can

Since the description of the technical features described above with reference to FIGS. 1 to 15 and 17 to 19 can be applied to FIG. 16 as it is, duplicate descriptions will be omitted.

Whether or not to use the calculation of the degree of occupancy of space in the embodiment (and the elevator car control based thereon) may be controlled through an API or a parameter of the API. For example, by turning on/off the corresponding parameter, the degree of space occupancy calculated according to the embodiment may or may not be used for elevator car control (eg, in the case of Off, a conventional weight sensor is used).

In the embodiment, the camera used to photograph the inside of the elevator car may be an AI Smart Camera that uses an FPGA SoC and enables (almost) real-time deep learning. AI Smart Camera can transmit the count number of objects and the calculated space occupancy level to the elevator car control panel through CAN communication. The captured image may be monitored through a gateway board and stored there.

The above-described elevator control system 130 may be a computer system (eg, including an FPGA SoC) included in such a camera (AI Smart Camera), and may be configured to include the above-described gateway board. In other words, the elevator control system 130 may be distinct from a control panel (and a system including the control panel) directly controlling the elevator car. The elevator control system 130 may transmit the calculated degree of space occupancy as a result of analyzing the image to the control panel (system including the control panel) as a signal for controlling the elevator car, and control of the elevator car is directly performed by the control panel. can be performed with

The system or device described above may be implemented as a hardware component, a software component, or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. may be permanently or temporarily embodied in Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (19)

In the elevator control method performed by the elevator control system,
Calculating a degree of space occupancy of the elevator car by an object inside the elevator car based on an image of the inside of the elevator car taken by a camera installed in the elevator car configured to allow a robot and a person to board; and
Based on the calculated degree of occupancy of space, generating a signal for controlling the elevator car to move to a position of the robot to board the robot in the elevator car.
including,
The step of calculating the degree of space occupancy,
detecting each of the objects riding in the elevator car from the image;
calculating a difference between an area of the floor of the elevator car and an area occupied by each of the objects or a ratio of an area occupied by each of the objects to an area of the floor; and
calculating the degree of occupancy of the space based on the calculated difference or ratio;
including,
Calculating the difference or ratio,
determining a position on the floor of each of the objects; and
Calculating an area occupied by each of the objects by applying to each of the objects a weight map defining a weight applied to each position on the floor.
Including, elevator control method. According to claim 1,
The camera is a wide-angle camera installed at the center of the ceiling of the elevator car or at the center of the corner of the ceiling on the opposite side of the door of the elevator car, and is configured to photograph the floor of the elevator car. delete delete delete According to claim 1,
The weight map is
It is defined as having a larger value of weight from the center of the floor toward the vertex of the floor,
Elevator control method, which is defined as having a different weight for each of the zones for the floor divided into a plurality of zones. According to claim 1,
The step of calculating the degree of space occupancy,
Calculating a first space occupation degree based on the calculated difference or ratio and a second space occupation degree based on the number of objects in the elevator car relative to the maximum number of passengers in the elevator car; and
Calculating the space occupation degree based on the first space occupation degree and the second space occupation degree
Including, elevator control method. In the elevator control method performed by the elevator control system,
Calculating a degree of space occupancy of the elevator car by an object inside the elevator car based on an image of the inside of the elevator car taken by a camera installed in the elevator car configured to allow a robot and a person to board; and
Based on the calculated degree of occupancy of space, generating a signal for controlling the elevator car to move to a position of the robot to board the robot in the elevator car.
including,
The step of calculating the degree of space occupancy,
Calculating a first space occupation degree based on the calculated difference or ratio and a second space occupation degree based on the number of objects in the elevator car relative to the maximum number of passengers in the elevator car; and
Calculating the space occupation degree based on the first space occupation degree and the second space occupation degree
including,
The degree of space occupancy is,
Calculated based on a sum of a value obtained by multiplying the first degree of space occupation by a first weight and a value obtained by multiplying the second degree of space occupation by a second weight,
The first weight is defined as a smaller value than the second weight, elevator control method. According to claim 1,
Detecting each of the objects,
Detecting each of the objects by detecting the head of each of the objects in the image,
The head of each object,
An elevator control method, which is detected using a learning model trained to detect the head of a person or robot from an image of a floor taken in a vertical direction. According to claim 1,
Calculating the difference or ratio,
segmenting each of the objects in the image to be distinguished from the floor;
identifying a portion occupied by the segmented object among floor portions corresponding to the floor; and
Calculating the ratio of the occupied part to the floor part as the ratio of the area occupied by the object to the area of the floor
Including, elevator control method. According to claim 1,
receiving a call for an elevator car from the robot or a robot control system that controls the robot;
Including more,
To generate the signal,
Generating a signal for controlling an elevator car determined to be boardable by the robot to move to a position of the robot based on the calculated degree of space occupancy
Including, elevator control method. According to claim 11,
The elevator control method of claim 1 , wherein the elevator car moving to the position of the robot is selected as one having the calculated degree of occupancy of space less than a predetermined value among a plurality of elevator cars. According to claim 11,
The call includes size information of the robot or required area information required for boarding in the elevator car of the robot,
Elevator control method, wherein it is determined whether or not the robot can board the elevator car based on the size information or the required area information and the calculated space occupation. According to claim 1,
Receiving a call for an elevator car for boarding of the robots from at least one of a plurality of robots or a robot control system that controls the robots
Including more,
To generate the signal,
Generating a signal for controlling an elevator car determined to be capable of being boarded by the robots to be moved to a position of the robots for boarding the robots based on the calculated degree of space occupancy
Including, elevator control method. According to claim 1,
The degree of space occupancy is calculated by analyzing the image based on the location information where the camera is provided, the maximum number of passengers for the elevator car, and the area information of the floor of the elevator car. Elevator control method. A computer program stored in a non-transitory computer readable recording medium to execute the method of any one of claims 1, 2 and 6 to 15 in the elevator control system, which is a computer system. A non-transitory computer readable recording medium in which a program for executing the method of any one of claims 1, 2, and 6 to 15 in the elevator control system, which is a computer system, is recorded. An elevator control system for controlling an elevator car configured to allow a robot and a person to board,
at least one processor implemented to execute computer readable instructions
including,
The at least one processor,
Calculate the degree of occupancy of space in the elevator car by an object riding in the elevator car based on an image of the interior of the elevator car taken by a camera provided in the elevator car;
Based on the calculated degree of occupancy of space, generating a signal for controlling the elevator car to move to a position of the robot to board the robot in the elevator car;
The at least one processor,
In order to calculate the degree of space occupancy, each of the objects riding in the elevator car is detected from the image, and the difference between the area of the floor of the elevator car and the area occupied by each of the objects or the area of the floor is calculated. Calculate the ratio of the area occupied by each of the objects for the object, and calculate the degree of occupancy of the space based on the calculated difference or ratio,
In calculating the difference or ratio, the position of each of the objects on the floor is determined, and a weight map defining a weight applied to each position on the floor is applied to each of the objects, thereby determining the position of each of the objects on the floor. An elevator control system that calculates the area each occupies. A method for calculating the degree of space occupancy for an elevator car, performed by a computer system, comprising:
Calculating a degree of space occupancy of the elevator car by an object inside the elevator car based on an image of the inside of the elevator car taken by a camera installed in the elevator car configured to allow a robot and a person to board; and
Transmitting the calculated degree of space occupancy to a system including a control panel for controlling the elevator car to move to a position of the robot to board the robot in the elevator car.
including,
The step of calculating the degree of space occupancy,
detecting each of the objects riding in the elevator car from the image;
calculating a difference between an area of the floor of the elevator car and an area occupied by each of the objects or a ratio of an area occupied by each of the objects to an area of the floor; and
calculating the degree of occupancy of the space based on the calculated difference or ratio;
including,
Calculating the difference or ratio,
determining a position on the floor of each of the objects; and
Calculating an area occupied by each of the objects by applying to each of the objects a weight map defining a weight applied to each position on the floor.
Including, how to calculate the degree of space occupancy.

Images