KR102600303B1 - Monitoring system and operating method thereof - Google Patents

Abstract

The present invention relates to a monitoring system and operating method for monitoring the appearance of objects around a vehicle, storing the objects when an event occurs, and transmitting the objects to a user terminal.
The monitoring system according to an embodiment of the present invention includes a first sub-device that detects the appearance of an object around the vehicle, initiates operation by receiving a first wake-up signal from the first sub-device, and tracks the object. It includes a second sub-device that counts the appearance time of the object, and a main device that starts operation upon receiving a second wake-up signal from the second sub-device and photographs and stores the object.

Description

Monitoring system and operating method thereof {MONITORING SYSTEM AND OPERATING METHOD THEREOF}

The present invention relates to a monitoring system and operating method for monitoring the appearance of objects around a vehicle, storing the objects when an event occurs, and transmitting the objects to a user terminal.

In general, black boxes are implemented in a way that captures and stores images when an impact or movement around the vehicle is detected while the vehicle is parked. The recently developed black box uses radar or lidar to detect a specific distance, and when it comes within a certain distance, the black box captures and stores images.

The above-mentioned background technology is technical information that the inventor possessed for deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be said to be known art disclosed to the general public before filing the application for the present invention.

Domestic Patent Publication No. 2015-0029139

The present invention was made to solve the above-mentioned problems and/or limitations, and the purpose of the present invention according to one aspect is to manage the vehicle using a detection unit such as radar or LiDAR, detect external hazards in advance, and notify the driver. I'm doing it.

In addition, in the case of existing radar or lidar detection or motion detection, the camera is activated from time to time on a road with many people, so the vehicle battery is drained quickly. However, the purpose of the present invention according to another aspect is to operate the camera when suspicious activity is detected to record the image. The aim is to reduce battery consumption by taking pictures.

A monitoring system according to an embodiment of the present invention includes a first sub-device that detects the appearance of objects around a vehicle; a second sub-device that starts operation upon receiving a first wake-up signal from the first sub-device to track the object and counts the appearance time of the object; and a main device that starts operation upon receiving a second wake-up signal from the second sub-device and photographs and stores the object.

The first sub-device may generate the first wake-up signal and transmit it to the second sub-device when an object appears around the vehicle.

The second sub-device may generate the second wake-up signal and transmit it to the main device when the tracking result of the object and the count result of the appearance time of the object satisfy event generation conditions.

The main device may initiate operation by the second wake-up signal and transmit the captured image of the object to the user terminal.

A method of operating a monitoring system according to an embodiment of the present invention includes detecting the appearance of an object around a vehicle by a first sub-device; Initiating an operation by a second sub-device upon receiving a first wake-up signal from the first sub-device to track the object and counting the appearance time of the object; and initiating an operation by receiving a second wake-up signal from the second sub-device to photograph and store the object.

The method may further include generating, by the first sub-device, the first wake-up signal when an object appears around the vehicle and transmitting the first wake-up signal to the second sub-device.

The method includes generating, by the second sub-device, the second wake-up signal and transmitting it to the main device when the tracking result of the object and the count result of the appearance time of the object satisfy event generation conditions; It may further include.

In the method, the main device may further include starting an operation by the second wake-up signal and transmitting the captured image of the object to the user terminal.

In addition to this, other methods for implementing the present invention, other systems, and computer programs for executing the methods may be further provided.

Other aspects, features and advantages in addition to those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to embodiments, the vehicle can be managed using a detection unit such as radar or LiDAR, and external risks can be detected in advance and notified to the driver.

Additionally, when suspicious activity is detected, the camera can be activated to record video to reduce battery consumption.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram schematically illustrating a monitoring system according to an embodiment of the present invention.
FIG. 2 is a diagram schematically illustrating the event occurrence conditions that initiate the operation of the main device in the personal monitoring system of FIG. 1.
FIG. 3 is a diagram schematically illustrating the battery consumption used for each device in the monitoring system of FIG. 1.
Figure 4 is a flowchart for explaining a method of operating a monitoring system according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but may be implemented in various different forms, and should be understood to include all conversions, equivalents, and substitutes included in the spirit and technical scope of the present invention. . The embodiments presented below are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. In describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. The above terms are used only for the purpose of distinguishing one component from another.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, identical or corresponding components are assigned the same drawing numbers and duplicate descriptions thereof are omitted. I decided to do it.

1 is a diagram schematically illustrating a monitoring system according to an embodiment of the present invention. Referring to FIG. 1, the monitoring system 1 may include a first sub-device 100, a second sub-device 200, and a main device 300.

The first sub-device 100 can detect the appearance of objects around the vehicle. As an example, the first sub-device 100 may include a laser sensor (not shown), a radar sensor (not shown), or a LiDAR device (not shown). Laser sensors are widely used in various fields, are widely used in security systems, and are also used to measure the position and direction of moving objects on assembly lines. Additionally, a laser sensor can be used to measure the distance from the sensor to an object, and can obtain information such as whether the object exists and, if so, the distance. Laser sensors are used in various fields such as home/building security, monitoring of intrusion into government offices/cultural properties, invisible fences in public facilities such as subways, and construction surveying. Recently, they have been installed in vehicles to sense the driving environment (roads, other vehicles, etc.). It is used in LIDAR (light detection and ranging) devices. A radar sensor transmits electromagnetic waves of the order of microwaves (microwaves, 10cm-100cm wavelength) to an object and receives electromagnetic waves reflected from the object to measure the distance, direction, altitude, etc. from the object.

The LiDAR device measures the time (i.e., time of flight) from the time the laser departs from the light source in the light emitting unit (not shown) until it is reflected by the object and enters the receiving unit (not shown). The distance can be calculated, and the sensor's precision can vary depending on how precisely the receiver measures the flight time. In addition, if the object is located at a long distance or the reflection coefficient is very small, the intensity of the laser reflected by the object and incident on the receiver is weak, so an error may occur in the measurement distance between the object and the sensor depending on the difference in the reflection coefficient of the object. These errors must be eliminated in fields that require high reliability, such as lidar devices applied to vehicles.

The second sub-device 200 starts operation by receiving the first wake-up signal from the first sub-device 100, tracks the object, and has a sub-controller capable of counting the object appearance time. It can be included. Here, the first wake-up signal may include a signal generated by the first sub-device 100 and transmitted to the second sub-device 200 when an object appears. For example, the first sub-device 100 transmits a detection signal to the second sub-device 200 regardless of whether an object appears, including a detection signal when the object does not appear and a detection signal when the object appears. Since the detection signals (first wake-up signals) are different from each other, the second sub-device 200 can detect whether an object appears from the detection signal from the first sub-device 100 and start the operation. The second sub-device 200 normally waits for operation in sleep mode, but when the first wake-up signal is received from the first sub-device 100, it is converted to active mode to track the object and the appearance time of the object. can be counted.

The main device 300 starts operation by receiving a second wake-up signal from the second sub-device 200, photographs and stores the object, or transmits the object photographed image to the user terminal through a communication network (not shown). Users can recognize situations occurring around their vehicle. Here, the second wake-up signal may include a signal generated by the second sub-device 200 and transmitted to the main device 300 when the object tracking result and the object appearance time count result satisfy the event generation conditions. .

In this embodiment, cases where the event creation conditions are met may include cases where the movement of an object within a predetermined distance around the vehicle continues for a predetermined time, and in this case, there may be a risk of vehicle theft. In addition, cases where the event creation conditions are met may include cases where an object enters within a predetermined distance around the vehicle and does not move for a predetermined period of time, and in such cases, there may be a risk of dangerous substances existing around the vehicle. In addition, cases that satisfy the event creation conditions may include cases where movement of an additional object occurs after an object enters very close to the vehicle, and such cases may include cases where the driver gets off after a contact accident. . Additionally, cases where the event creation conditions are met may include a case where an object approaches the vehicle quickly and suddenly stops, and such a case may include a case where the object collides with the vehicle. The main device 300 normally waits for operation in sleep mode, but when a second wake-up signal is received from the second sub-device 200, it is converted to active mode to capture and store an object or capture a captured image of an object. It can be transmitted to the user terminal through a communication network.

In this embodiment, the main device 300 may include a main control unit 310, a camera 320, a memory 330, and a communication unit 340.

The main control unit 310 operates the main unit 300 in sleep mode, and when receiving a second wake-up signal from the second sub-unit 200, operates the main unit 300 in active mode, and the camera 320 , the operations of the memory 330 and the communication unit 340 can be controlled.

The main control unit 310 is a type of central processing unit and can provide various functions by running control software mounted on an internal memory (not shown). The main control unit 310 may mean, for example, a data processing device built into hardware that has a physically structured circuit to perform a function expressed as a code or command included in a program. Examples of data processing devices built into hardware include a microprocessor, central processing unit (CPU), processor core, multiprocessor, and application-specific integrated (ASIC). circuit) and FPGA (field programmable gate array), but the scope of the present invention is not limited thereto.

The camera 320 may operate under the control of the main control unit 310 and include one or more cameras arranged on the top/bottom/left/right side to capture images around the vehicle. Here, the camera 320 can photograph a subject within the shooting area using, for example, a complementary metal-oxide semiconductor (COMS) module (not shown) or a charge coupled device (CCD) module (not shown), and input video frames. It is provided as a COMS module or CCD module through a lens (not shown), and the COMS module or CCD module can convert the optical signal of the subject that has passed through the lens into an electrical signal and output it. The camera 320 is equipped with an image signal processor (not shown) to reduce noise in captured image frames, perform gamma correction, color filter array interpolation, and color processing. Image signal processing to improve image quality, such as color matrix, color correction, and color enhancement, can be performed. In addition, the image signal processing unit can functionally perform color processing, blur processing, edge emphasis processing, image analysis processing, image recognition processing, and image effect processing. Face recognition, scene recognition processing, etc. can be performed through image recognition processing. For example, luminance level adjustment, color correction, contrast adjustment, outline emphasis adjustment, screen splitting processing, creation of character images, etc., and image compositing processing can be performed.

The memory 330 operates under the control of the main control unit 310 to store or output an object image captured by the camera 320. This memory 330 may include internal memory and/or external memory, volatile memory such as DRAM, SRAM, or SDRAM, one time programmable ROM (OTPROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, Non-volatile memory, such as NAND flash memory, or NOR flash memory, SSD. It may include a flash drive such as a compact flash (CF) card, SD card, Micro-SD card, Mini-SD card, Xd card, or memory stick, or a storage device such as an HDD.

The communication unit 340 may provide a communication interface necessary to provide an object image captured by the main device 300 to a user terminal in conjunction with a communication network. Here, the communication network may mean a communication network that provides a connection path so that the main device 300 can transmit and receive certain information after connecting to the user terminal. Communication networks include, for example, wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), or wireless networks such as wireless LANs, CDMA, Bluetooth, and satellite communications. may encompass, but the scope of the present invention is not limited thereto. Additionally, the communication network 500 can transmit and receive information using short-distance communication and/or long-distance communication. Here, short-range communication may include Bluetooth, RFID (radio frequency identification), infrared communication (IrDA, infrared data association), UWB (ultra-wideband), ZigBee, and Wi-Fi (wireless fidelity) technology, and long-distance communication may include Communications may include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), and single carrier frequency division multiple access (SC-FDMA) technologies. You can.

In addition, the main device 300 randomly starts operation at the request of the user terminal through the communication unit 340 and sends the image around the vehicle captured by the main device 300 or the object captured image stored in the memory 330 to the user terminal. You can also send it to . When receiving a request signal from the user terminal, the first sub-device 100 and the second sub-device 200 may maintain the sleep mode state and only the main device 300 may enter the active mode state.

In this embodiment, the user terminal refers to a desktop computer, smartphone, laptop, tablet PC, smart TV, mobile phone, PDA (personal digital assistant), laptop, media player, micro server, etc., which are not shown but are operated by the user. It may be, but is not limited to, a global positioning system (GPS) device, e-reader, digital broadcasting terminal, navigation, kiosk, MP3 player, digital camera, home appliance, and other mobile or non-mobile computing devices. Additionally, the user terminal may be a wearable terminal such as a watch, glasses, hair band, or ring equipped with a communication function and data processing function. The user terminal is not limited to the above, and any terminal capable of web browsing can be used without limitation. In this embodiment, the user terminal may be equipped with an application and/or program that controls the monitoring system 1.

In an optional embodiment, the second sub-device 200 may enter sleep mode when the first wake-up signal is not received from the first sub-device 100 for a certain period of time, and the main device 300 may enter the sleep mode ( If the second wake-up signal is not received from 200) for a certain period of time, the device may enter sleep mode.

FIG. 2 is a diagram schematically illustrating the event occurrence conditions that initiate the operation of the main device in the personal monitoring system of FIG. 1. Referring to FIG. 2A, as a result of the object tracking by the second sub-device 200, when an object enters within a predetermined distance around the vehicle and movement occurs around the vehicle for more than a predetermined period of time, the second sub-device 200 A second wake-up signal may be generated and transmitted to the main device 300, and the main device 300 may initiate an operation in response to receiving the second wake-up and perform image capture, storage, and/or transmission. Referring to FIG. 2b, the object tracking result of the second sub-device 200 may include a case in which movement of an additional object occurs after the object enters very close to the vehicle, and this case may include a case where the driver gets off after a contact accident. Cases may be included. In this case, when an additional object is detected, the second sub-device 200 generates a second wake-up signal and transmits it to the main device 300, and the main device 300 operates in response to receiving the second wake-up. You can start to capture, store, and/or transmit images.

FIG. 3 is a diagram schematically illustrating the battery consumption used for each device in the monitoring system of FIG. 1. Referring to FIG. 3, in normal times, only the first sub-device 100 of the monitoring system 1 can start operation to monitor the surroundings of the vehicle (monitoring phase). When the first sub-device 100 detects the appearance of an object, it transmits the first wake-up signal to the second sub-device 200, so that the second sub-device 200 starts operation, tracks the object, and counts time. It can be done (object tracking stage). If the second sub-device 200 satisfies the event generation conditions as a result of object tracking, a second wake-up signal is transmitted to the main device 300, so that the main device 300 starts operation, and captures, stores, and/or images of the object. Or it can be transmitted (video storage stage). From Figure 3, the battery consumption can be seen in the order of the monitoring phase, object tracking phase, and image storage phase. Battery consumption can be reduced by starting the operation of the main device 300 and recording images only when the event creation conditions are met. .

Figure 4 is a flowchart for explaining a method of operating a monitoring system according to an embodiment of the present invention. In the following description, parts that overlap with the description of FIGS. 1 to 3 will be omitted.

Referring to FIG. 4, in step S410, the first sub-device 100 monitors the appearance of objects around the vehicle. Here, the first sub-device 100 may include a radar sensor or LiDAR device. Here, only the first sub-device 100 is operating, and the second sub-device 200 and the main device 300 are in sleep mode.

In step S420, when an object appears around the vehicle, the first sub-device 100 generates a first wake-up signal and transmits it to the second sub-device 200.

In step S430, the second sub-device 200 starts operation when it receives the first wake-up signal from the first sub-device 100.

In step S440, the second sub-device 200 tracks objects around the vehicle and counts time.

In step S450, if the event generation condition is satisfied, the second sub-device 200 generates a first wake-up signal and transmits it to the main device 300. In this embodiment, cases where the event creation conditions are met may include cases where the movement of an object within a predetermined distance around the vehicle continues for a predetermined time, and in this case, there may be a risk of vehicle theft. In addition, cases where the event creation conditions are met may include cases where an object enters within a predetermined distance around the vehicle and does not move for a predetermined period of time, and in such cases, there may be a risk of dangerous substances existing around the vehicle. In addition, cases that satisfy the event creation conditions may include cases where movement of an additional object occurs after an object enters very close to the vehicle, and such cases may include cases where the driver gets off after a contact accident. . Additionally, cases where the event creation conditions are met may include a case where an object approaches the vehicle quickly and suddenly stops, and such a case may include a case where the object collides with the vehicle.

In step S460, the main device 300 starts operation when receiving the second wake-up signal from the second sub-device 200.

In step S470, the main device 300 captures and stores an object image or transmits the object image to the user terminal.

Embodiments according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. At this time, the media includes magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROM. , RAM, flash memory, etc., may include hardware devices specifically configured to store and execute program instructions.

Meanwhile, the computer program may be designed and configured specifically for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer programs may include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

In the specification (particularly in the claims) of the present invention, the use of the term “above” and similar referential terms may refer to both the singular and the plural. In addition, when a range is described in the present invention, the invention includes the application of individual values within the range (unless there is a statement to the contrary), and each individual value constituting the range is described in the detailed description of the invention. It's the same.

Unless there is an explicit order or statement to the contrary regarding the steps constituting the method according to the invention, the steps may be performed in any suitable order. The present invention is not necessarily limited by the order of description of the above steps. The use of any examples or illustrative terms (e.g., etc.) in the present invention is merely to describe the present invention in detail, and unless limited by the claims, the scope of the present invention is limited by the examples or illustrative terms. It doesn't work. Additionally, those skilled in the art will recognize that various modifications, combinations and changes may be made depending on design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the scope of the claims are within the scope of the spirit of the present invention. It will be said to belong to

1: Monitoring system
100: first sub-device
200: second sub device
300: Main device

Claims (9)

A first sub-device that detects the appearance of objects around the vehicle;
a second sub-device that starts operation upon receiving a first wake-up signal from the first sub-device to track the object and counts the appearance time of the object; and
A main device that starts operation upon receiving a second wake-up signal from the second sub-device to photograph and store the object,
The second sub-device,
If the tracking result of the object and the count result of the appearance time of the object satisfy event generation conditions, the second wake-up signal is generated and transmitted to the main device,
A monitoring system that determines whether the event generation conditions are satisfied in response to the movement of the object and the entry speed of the object that has entered a predetermined distance from the vehicle. The method of claim 1, wherein the first sub-device:
A monitoring system that generates the first wake-up signal and transmits it to the second sub-device when an object appears around the vehicle. The method of claim 1, wherein the second sub-device:
When the movement of the object continues for a predetermined time within a predetermined distance from the vehicle, when the object does not move for a predetermined time after entering within a predetermined distance from the vehicle, after the object moves closer than the predetermined distance from the vehicle A monitoring system that determines that the event generation conditions are satisfied in at least one of the following cases: when additional movement occurs and when the object approaches the vehicle within a predetermined distance within a predetermined time and then stops. The method of claim 1, wherein the main device:
A monitoring system that starts operation by the second wake-up signal and transmits the captured image of the object to the user terminal. Detecting, by a first sub-device, the appearance of an object around the vehicle;
Initiating an operation by a second sub-device upon receiving a first wake-up signal from the first sub-device to track the object and counting the appearance time of the object;
Generating, by the second sub-device, a second wake-up signal and transmitting it to the main device when the tracking result of the object and the count result of the appearance time of the object satisfy event generation conditions; and
Initiating an operation upon receiving the second wake-up signal from the second sub-device to photograph and store the object,
A method of operating a monitoring system that determines, by the second sub-device, whether the event generation condition is satisfied in response to the movement of the object that has entered a predetermined distance from the vehicle and the entry speed of the object. According to clause 5,
A method of operating a monitoring system further comprising: generating, by the first sub-device, the first wake-up signal when an object appears around the vehicle and transmitting the first wake-up signal to the second sub-device. According to clause 5,
When the movement of the object continues for a predetermined time within a predetermined distance from the vehicle by the second sub-device, and the object does not move for a predetermined time after entering within a predetermined distance from the vehicle, the object is A monitoring system that determines that the event generation conditions are satisfied in at least one of the following cases: when additional movement occurs after getting closer to the vehicle than the predetermined distance, and when the object stops after approaching the vehicle within a predetermined distance within a predetermined time. How it works. According to clause 5,
The method of operating a monitoring system further comprising: the main device starting operation in response to the second wake-up signal and transmitting the captured image of the object to a user terminal. A computer program stored in the computer-readable recording medium to execute the method of any one of claims 5 to 8 using a computer.

Images