KR20220032332A - System for estimating flood level based on CCTV device - Google Patents

Abstract

The present invention relates to a CCTV device-based flood level estimation system, which includes: a reference image database unit storing reference image data in association with reference geometry information; a search and selection unit acquiring CCTV geometry information including second external geometry data related to image data acquired from a CCTV device and referring to at least reference location data and two-dimensional location data to search for and select the image data matchable with the reference image data; a matching unit matching the reference image data and the selected image data with each other; a geometric transformation unit geometrically transforming the CCTV geometry information of the selected image data by geometric transformation modeling referring to the reference image data to generate calibration CCTV geometry information; a ground surface space information extraction unit using CCTV image information generated from the image data of the CCTV device based on the calibration CCTV geometry information and matching information at an image information generation unit to extract ground surface space information related to a ground surface included in the image data of the CCTV device in a region of interest and an object adjacent to the ground surface; and a flood level estimation unit estimating, in the event of a flood, the flood level from the image data of the region of interest based on the ground surface space information.

Description

CCTV device-based flood level estimation system {System for estimating flood level based on CCTV device}

The present invention relates to a CCTV device-based flood level estimation system and method in which a CCTV image of a flood-damaged area obtained from a CCTV device in which geometric information such as location and posture is converted and spatial information of a flood-damaged area is linked.

With the development of communication technology, infrastructure technology and camera technology, CCTV is installed and used in many places, and the national government and local governments are also installing and using CCTV for various purposes such as traffic management, disaster preparedness, and crime prevention.

Accordingly, a large amount of CCTV is installed and used in many areas, and the situation at the location can be monitored by managing the address information or longitude and latitude information of the CCTV together.

In general, in order to access the CCTV, the CCTV URL is accessed and analyzed in real time through field values in the table, such as address values or location values, in a database organized in a table format.

There are many research results on technologies for recognizing and analyzing objects and situations existing in CCTV by applying image information-based deep learning and machine learning technologies to most of the conventional analysis technologies.

CCTV infrastructure combined with such deep learning and machine learning technologies is being tried in various ways as part of the smart city project.

However, there are cases in which incorrect information is entered or data is lost due to insufficient data management of CCTV-related information.

Common CCTV standard data is often accompanied by a location error of several meters to several tens of meters in the case of location information, and some CCTV data is often missing location information or the location is vaguely described.

In addition, direction information of CCTV is required for precise spatial information analysis, but there is a problem in that there is no direction information, so there is a limit in providing a map service in connection with spatial information.

Since there is a limit in the case of organically analyzing the target area regionally or regionally, such a limitation requires agents who monitor the screen images in real time at a CCTV control center, etc., and the infrastructure for analyzing the image. cost is needed For this reason, it is acting as an obstacle to the implementation of a smart city.

Moreover, since it is difficult to access the area when a flood occurs, there is a limit to the implementation of the flood level estimation for establishing a disaster response plan using CCTV images due to the above-mentioned limitations.

On the other hand, in order to provide spatial information service through CCTV images, location and posture information of CCTV is important. The location and posture information is important data to determine the location on the map of the objects expressed in the image.

In order to overcome this limitation, it is necessary to establish reference data to correct the position and posture of CCTV, and to extract spatial information of objects detected in the image through the corrected information. However, since CCTVs are sporadically distributed, collecting reference data also requires a lot of manpower and cost. In addition, in order to connect the collected reference data with CCTV image information, the image coordinate position of the reference data must be registered in the CCTV image. However, this operation also consumes a lot of manpower and cost, and has a disadvantage in that it is difficult to grasp the exact position and posture value when the zoom, pan, and tilt functions are activated in the CCTV.

Therefore, in order to implement the estimation of the flood level in the flood-damaged area using the CCTV image, geometric information such as the position and posture of the CCTV image is corrected, and spatial information of objects detected in the CCTV image through the corrected information is corrected. There is a continuous need for a base technology that can be extracted.

The technical problem to be achieved by the present invention is to match the video data of a CCTV device having inaccurate or missing position and posture-related geometric data in some areas with the reference image information of the reference image data generated from a multi-sensor platform composed of an observation sensor and a position sensor An object of the present invention is to provide a CCTV device-based flood level estimation system and method for generating spatial information with convenience and high precision from image data acquired with a CCTV device by performing geometric transformation and the like.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention for achieving the above technical problem, a CCTV device-based flood level estimation system is. A reference image database unit for storing reference image data in association with reference position data and reference geometry information including first external geometric data having first posture data defining a viewing direction set to generate a reference image, and a CCTV device Obtain CCTV geometric information including second external geometric data having at least two-dimensional position data in relation to the image data acquired from A search selection unit for searching and selecting the image data, a matching unit for mutually matching between the reference image data and the selected image data, and the CCTV of the selected image data by geometric transformation modeling referring to the reference image data A geometric transformation unit for generating corrected CCTV geometry information including position data for correction with higher accuracy than the two-dimensional position data by geometric transformation of geometric information, and the CCTV geometry information and matching information for correction in the image information generation unit, based on the Using the CCTV image information generated from the image data of the CCTV device, extracting the ground surface spatial information about the ground surface included in the video data of the CCTV device in the region of interest (ROI) and the object adjacent to the ground surface and a surface spatial information extracting unit, and a submersion level estimator configured to estimate a flood level from the image data of the region of interest when a flood occurs based on the ground surface spatial information.

In another embodiment, the ROI may be selected to include an object having a shape formed vertically from the ground surface, and the ground surface spatial information may include spatial information along a vertical direction of the object.

In another embodiment, the submergence level estimator estimates the submersion level of the region of interest by comparing the spatial information of the object included in the image data of the region of interest obtained in real time when a flood occurs, against the spatial information on the ground, The inundation level is estimated in real time by the difference between the three-dimensional spatial information of the object exposed at the same level as the water level due to the flood and the three-dimensional spatial information of the surface spatial information, or the area of interest in normal times related to the surface spatial information By analyzing the difference between the object image and the object image of the region of interest when a flood occurs, the submergence level can be estimated in real time.

In another embodiment, the two-dimensional position data includes the two-dimensional coordinates of the CCTV device or the name of the space in which the CCTV device is installed, and the reference position data is more accurate than the two-dimensional position data of the first external geometric data. is composed of high two-dimensional position data or three-dimensional position data, and when the reference position data is two-dimensional position data, the CCTV image information includes two-dimensional position coordinates, and the reference position data is three-dimensional position data In the case of , the CCTV image information may include three-dimensional position coordinates.

In another embodiment, the second external geometric data further includes second posture data defining a viewing direction set to generate the image, and the search selection unit is based on the reference position data and the two-dimensional position data. A candidate group of the image data that can be compared with the reference image data may be searched, and the image data having the highest degree of matching with the reference image data may be selected from the candidate group by referring to the first and second posture data.

In another embodiment, the reference image data includes image information and point cloud information obtained from an external mobile platform having at least one of an image capturing sensor and a laser scanning sensor and a positioning sensor through a geometric model between the sensors. It may be external image data fused by matching and correcting, or corrected image data already stored as CCTV image information after being corrected with the CCTV geometry information for correction.

In another embodiment, when at least one of the reference image and the image data is obtained from a wide-area image sensor generating a distorted image, the reference image and the image data from which the distorted image is generated are the distorted images. When plane image data converted into a plane image is used and the reference image uses the plane image data, the first posture data may be defined as a viewing direction of the plane image data.

In another embodiment, when the reference image originates from a laser scanning sensor, the reference image uses virtual image data obtained by converting a point cloud obtained from the laser scanning sensor, and the reference position data is It may be defined as position data of the virtual image data, and the first posture data may be defined as a viewing direction of the virtual image data.

In another embodiment, the matching unit extracts and compares feature geometries from the selected image data and the reference image data to mutually match commonly estimated feature geometries, or is at least similar to a template specified in the reference image data. The selected image data may be searched for and matched according to characteristic correlation analysis between the templates, the matching information may be the feature geometry or the template, and the feature geometry or the template may be stored in a reference image database unit.

In another embodiment, by randomly sampling the sample feature geometries of the video data linked with the CCTV geometry information for correction, the amount of separation between the sample feature geometries and the corresponding feature geometries of the reference data is minimized An optimal value estimator for estimating the geometry as a matching optimal value, and a geometry information estimator for estimating the geometrical transformed CCTV geometry information for correction in relation to the feature geometry estimated as the matching optimal value as the final corrected CCTV geometry information further comprising, Among the information based on generation of the CCTV image information, the correction CCTV geometry information may use the final correction CCTV geometry information.

In another embodiment, the reference geometric information is composed of three-dimensional position data as the reference position data and the first external geometric data having the first posture data, and is related to a partial position of the reference image data and the reference position data When the precision of the geometric information is lower than the threshold, the image information generating unit converts the corrected image data of the CCTV image information corresponding to the low precision reference image data into a two-dimensional geometric model, then the two-dimensional converted image Estimating the object height through pixel data analysis within the data, analyzing the object height assuming the height of the image sensor of the CCTV device as a specific height, or specifying posture data related to the viewing direction of multiple image sensors of the CCTV device By defining a value and estimating the object height using a trigonometric model, and estimating three-dimensional position data by reflecting the estimated object height in the corrected CCTV geometric information, the CCTV image information corresponding to can be generated. .

The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows, and do not limit the scope of the present disclosure.

According to the present invention, image data of a CCTV device having an inaccurate or missing position and posture-related geometric data in some areas is matched with the reference image information of the reference image data generated from a multi-sensor platform composed of an observation sensor and a positioning sensor to obtain a geometry By performing conversion correction, etc., it is possible to generate image data acquired with a CCTV device as spatial information with convenience and high precision.

In addition, by linking the CCTV image of the flood-damaged area obtained from the CCTV device in which geometric information such as location and posture has been converted with the spatial information of the flood-damaged area, it is possible to analyze The flood level can be accurately estimated.

In addition, it is possible to link and analyze the corrected position and posture information and the image information obtained from CCTV to transform the object information expressed in the image into spatial information and link it with the map service.

In addition, by processing observation data obtained from low-cost sensors excluding expensive sensors such as lidar when constructing the reference image data, it is possible to achieve reduction in construction cost.

In addition, by storing the corrected geometric information such as the position and posture of the CCTV on the reference image database unit, and using it when generating 3D spatial information of image data of other CCTVs in a similar area, various types of reference image data can be used

In addition, the effect derived through the configuration that can be understood by those skilled in the art through the present specification is not excluded. The effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be.

1 is a configuration diagram of a CCTV device-based flood level estimation system, a CCTV device communicating therewith, and a CCTV controller according to an embodiment of the present invention.
2 is a flowchart of a method for generating CCTV image information for image data of a CCTV device and for estimating a submersion level implemented in a CCTV device-based flood level estimation system according to an embodiment of the present invention.
3 is a diagram illustrating an example of reference image information stored in a reference database unit.
4 is a diagram illustrating types of data stored in first external geometric data and first internal geometric data when reference image information stored in the reference database unit is acquired from an image capturing sensor.
5 is a diagram illustrating a process of transforming a geometric shape in order to search for and extract a feature geometry when reference image information stored in the reference database unit is obtained from aerial observation data.
6 is a flowchart of a first process of converting image data acquired from a wide area image sensor into flat image data by removing distortion.
7 and 8 are diagrams schematically illustrating a first process of image distortion removal and flat image data conversion.
9 is a flowchart illustrating a second process of converting image data acquired from a wide area image sensor into flat image data by removing distortion.
10 is a diagram schematically illustrating a second process of image distortion removal and flat image data conversion.
11 is a diagram illustrating a form in which virtual image data obtained by converting a point cloud obtained from a laser scanning sensor in a reference image database unit is stored.
12 is a flowchart related to another embodiment of search selection in a process of generating spatial information for image data of a CCTV device.
13 is a diagram illustrating a matching process using feature geometry in the matching unit.
14 is a flowchart illustrating a matching process using a characteristic correlation analysis between templates in the matching unit.
15 is a diagram illustrating a process of calculating two-dimensional position data of image data and three-dimensional position data of reference image data corresponding to the two-dimensional position data of image data.
16 is a flowchart illustrating a process of estimating the height of an object of CCTV image data in the image information generating unit.
17 is a view for explaining a process of estimating an object height through pixel data analysis in CCTV image data.
18 is a diagram illustrating an example of estimating the submersion level implemented in the system for estimating the submersion level according to an embodiment of the present invention.
19 is a flowchart of a method for generating and updating spatial information implemented in a CCTV device-based flood level estimation system.
20 is a diagram illustrating an example of generating new spatial information including final corrected CCTV geometry information and object information in CCTV image information.
21 is a diagram for explaining a process of updating spatial information.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings and the content to be described later. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed subject matter may be thorough and complete, and that the spirit of the present invention may be sufficiently conveyed to those skilled in the art. Like reference numerals refer to like elements throughout. On the other hand, the terms used herein are for the purpose of describing the embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, “comprises” and/or “comprising” means that the stated component, step, operation and/or element is the presence of one or more other components, steps, operations and/or elements. or addition is not excluded.

In addition, "unit" or "module" generally refers to logically separable software (computer, program), hardware, and the like parts. Accordingly, the module in the present embodiment refers to not only a module in a computer and a program but also a module in a hardware configuration. Therefore, the present embodiment provides a computer program for making them function as modules (a program for causing the computer to execute each step, a program for making the computer function as each means, a program for making the computer to realize each function) ), systems and methods are also explained. However, for convenience of explanation, "save", "save", and phrases equivalent to these are used, but when the embodiment is a computer program, these words are stored in a storage device or stored in a storage device It means to control as In addition, although "parts" and modules may correspond to functions one-to-one, in implementation, one module may be configured as one program, multiple modules may be configured as one program, or conversely, one module may be configured with multiple programs. do. In addition, a plurality of modules may be executed by one computer, and one module may be executed by a plurality of computers by a computer in a distributed or parallel environment. In addition, another module may be included in one module. In addition, hereinafter, "connection" is employed not only for physical connection but also for logical connection (exchange of data, instruction, reference relationship between data, etc.). "Predetermined" means that it is determined before the target processing, and before the processing according to the present embodiment is started, as well as after the processing according to the present embodiment is started, before the target processing, the It is used with the meaning of being determined according to the situation or state at the time, or by the situation or state up to that time.

In addition, a system or device means a plurality of computers, hardware, devices, etc. connected by communication means such as a network (including a one-to-one correspondence) and configured by one computer, hardware, device, etc. cases to be realized are included. "Device" and "system" are used as mutually synonymous terms. Of course, "system" does not include things that are nothing more than social "organizations" (social systems) that are artificial decisions.

In addition, for each processing by each unit or each module, or when a plurality of processing is performed within each unit or module, target information is read and inputted from the storage device for each processing, and after performing the processing, the processing result is displayed writing to the memory device. Therefore, the description of read input from the storage device before processing and writing to the storage device after processing may be omitted in some cases. Note that, as the storage device here, a hard disk, random access memory (RAM), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like may be included.

Hereinafter, with reference to FIG. 1, corrected image data of a CCTV device generated by matching the image data of the two-dimensional position information acquired from the CCTV device according to an embodiment of the present invention with the reference image data of the reference image database unit and performing a predetermined process A description will be given of a CCTV device-based flood level estimation system for generating CCTV image information based on .

1 is a configuration diagram of a CCTV device-based flood level estimation system, a CCTV device communicating therewith, and a CCTV controller according to an embodiment of the present invention.

The system 100 transmits two-dimensional location data of the CCTV device 102 and/or image data included in the image data and attribute information of spatial objects such as terrain and features photographed by the CCTV device 102 to the CCTV controller ( 104). The CCTV controller 104 receives information such as the viewing direction of the image sensor 106 of the CCTV device 102 estimated at the time of shooting as attribute information, and geometric parameters unique to the image sensor 106 .

First, the CCTV device 102 and the CCTV controller 104 will be described. The CCTV device 102 may include an image sensor 106, a communication unit 108, and a control unit 110 for controlling and managing them. The filing 104 may include an attribute information management unit 111 , an image storage unit 112 , and a control/communication unit 114 .

The CCTV device 102 transmits the image data captured through the image sensor 106 to the CCTV controller 104 by the communication unit 108 . The CCTV controller 104 is a device that transmits information in which two-dimensional location data of CCTV stored as attribute information in received image data is combined to the system 100 through a wired/wireless network. As described above, the attribute information may further include a photographing time of the image sensor 106 , a viewing direction of the image sensor 106 , and unique geometric parameters, and the image data in which the viewing direction and geometric parameters are further combined transmitted to the system 100 .

The image sensor 106 is mounted on the CCTV device 102 and captures surrounding objects, for example, terrain and features, as images to acquire image data and time information for images at the time of image data acquisition. Can be provided with a plurality of sensors. there is. The image sensor 106 may be a flat image sensor, a wide-area image sensor having a wider field of view than a planar image sensor, or a surveying or non-surveying camera, but is not limited thereto. The wide-area image sensor generates a stereo image in which images having different viewing directions are combined, or a panoramic image in which an image of a camera disposed within a predetermined viewing angle is combined, for example, a camera having a fisheye lens to generate a fisheye lens image Alternatively, it may be a multi-camera system that outputs an omnidirectional image by a multi-camera using a plurality of cameras. The module on which the image sensor 106 is mounted may further include zoom, pan, and tilt functions.

In the case where the image data originates from a wide-area image sensor that generates a distorted image, the image data is retrieved by the search selection unit 118, which will be described later. Planar image data generated by converting a distorted image into planar image data may be used, and the detailed conversion process will be described later.

The property information management unit 111 of the CCTV controller 104 manages the property information of a plurality of CCTV devices 102 in the form of a table recorded in numerical or text format, and transfers the property information to the video data by the control/communication unit 114 combine

The attribute information includes two-dimensional location data of the CCTV device 102, and the two-dimensional location data may be the two-dimensional coordinates of the CCTV device 102, or the name of the space in which the CCTV device is installed as rough information rather than coordinates. there is. The attribute information may additionally have a photographing time of the image sensor 106 . In some cases, the attribute information may additionally include a viewing direction such as a photographing direction and an angle of the image sensor 106, a viewing angle, and a geometric parameter defined in the image sensor 106, but typical attribute information is generally two-dimensional. Since it has only position data, a process of selecting image data by distinguishing between a case having only two-dimensional position data and a case including a viewing direction and a geometric parameter will be described later, respectively.

Posture data having a viewing direction and viewing angle (hereinafter referred to as 'second posture data'), two-dimensional position data, and shooting time may constitute external geometric data (hereinafter referred to as 'second external geometric data'), , the geometric parameters defined in the image sensor 106 may constitute internal geometric data (hereinafter, referred to as 'second internal geometric data').

The second external geometric data is an error of observation data calculated based on a parameter value that is changed whenever image information of the image sensor 106 is acquired due to the position and posture of the image sensor 106 . It can be calculated through the formula or modeling of. When the image data originates from the wide-area image sensor, the 2D position data may use 2D position data of the planar image data, and the second posture data may be defined as a viewing direction and a viewing angle of the planar image data.

The second internal geometric data is an eigenvalue of the image sensor 106 itself, and is a geometric parameter related to an error of observation data of the image sensor 106 . The geometric parameter of the image sensor 106 may include any one or more of a focal length, a principal point position, a lens distortion parameter, and a sensor format size.

The control/communication unit 114 generates CCTV geometric information including at least two-dimensional position data and external geometric data having a shooting time in the image data acquired from the image sensor 106 and controls it to be transmitted to the system 100 or , it is possible to receive specific data from the system 100 and perform necessary processing.

The system 100 includes a reference image database unit 116 , a search selection unit 118 , a matching unit 120 , a geometric transformation unit 122 , a matching optimal value estimation unit 124 , and a CCTV geometric information estimation unit 126 . , an image information generating unit 128 , an object information extracting unit 130 , a spatial information generating unit 132 , a spatial information updating unit 134 , a spatial information database unit 136 , a surface spatial information extracting unit 138 , and It may include a flood level estimation unit 140 .

The reference image database unit 116 includes reference position data of a reference image, an acquisition time of the reference image, a viewing direction set for generating the reference image, and first external geometric data having first posture data defining a viewing angle. The reference image data is stored in association with the geometric information.

The reference position data may be composed of two-dimensional position data or three-dimensional position data with higher accuracy than the two-dimensional position data of the first external geometric data, rather than the schematic information of the two-dimensional position data of the CCTV device 102, It may be two-dimensional position data of the XY coordinate system or three-dimensional position data of the XYZ coordinate system. The viewing direction and viewing angle are substantially the same as those described for the image sensor 106 . Also, the reference geometric information may further include first internal geometric data based on a geometric parameter defined in a sensor configured to generate a reference image.

The reference image is an image obtained from the outside and includes at least reference position data, and is processed into reference image data that can be contrasted with the image data obtained from the CCTV device 102 . The sensor initially used to generate the reference image may be at least one of an image acquisition device and a laser scanning sensor (eg, Lidar) having higher positioning accuracy and image realization performance than the CCTV device 102 . The reference geometric information associated with the reference image includes color information such as RGB data, first external geometric data, and first internal geometric data for each pixel coordinate, and the reference image combined with the reference geometric information is stored as reference image data can be

The reference image data matches and corrects image information and point cloud information obtained from an external mobile platform having at least one of an image capturing sensor (image acquisition device) and a laser scanning sensor and a positioning sensor through a geometric model between the sensors. It may be the corrected image data of the existing CCTV device already stored in the image information generating unit 128 and/or the spatial information database unit 136 as the external image data fused by the there is.

When the sensor initially used to generate the reference image is an image capturing sensor, the first internal geometric data is substantially the same as the image sensor 106 described in the second internal geometric data. In the case of an image capturing sensor, since the projection center and direction used to project the three-dimensional shape of the object in two dimensions are related to the reference position data and the viewing direction of the reference image, the projection center and direction are combined with the first external geometric data It may be stored as reference image data. The reference geometry information of the reference image data generated by the image capturing sensor may include two-dimensional position data or three-dimensional position data as reference position data.

When the reference image is aerial observation data, since it has a different geometry from ground observation image data such as the CCTV device 102 , the aerial observation data may be converted into a view form in a specific direction. For example, each pixel of the aerial observation data may be converted into an orthogonal image form having a horizontal geometry and employed as a reference image.

When the reference image originates from a wide-area image sensor that generates a distorted image, the reference image uses planar image data generated by converting the distorted image into planar image data, and the reference position data uses position data of the planar image data. and the first posture data may be defined as a viewing direction, a viewing angle, etc. of the planar image data. A detailed conversion process will be described later.

When the reference image originates from the laser scanning sensor, the reference image may use virtual image data obtained by converting a point cloud obtained from the laser scanning sensor. The 3D position data is a 3D position of the virtual image data. It is defined as data, and the first posture data may be defined as a viewing direction of the virtual image data. Transformation of the point cloud may be performed by virtual camera modeling in consideration of the reflection intensity and elevation value of the point cloud. When the image sensor is additionally involved in generating the reference image, color data obtained by the image sensor when the point cloud is converted may be included in the virtual image data.

When the first used sensor is a laser scanning sensor, the first internal geometric data may be, for example, at least one of an incident angle of each laser, a distance scale, a distance direction offset, and an axial direction offset. The point cloud information acquired by the laser scanning sensor may be expressed as a set of points including all or part of three-dimensional coordinate information, laser intensity information, color information, class information, time information, and sensor observation information.

When a reference image is acquired from an external mobile platform having a multi-sensor such as an imaging sensor, a laser scanning sensor, and a positioning sensor, matching and correction of the applied model between the sensors is performed by the image sensor and the laser scanning sensor (LIDA). ), an external geometric model of LiDAR for combining data with a positioning sensor, and an external geometric model of an image capturing sensor for combining data with a positioning sensor. Each geometric model may be defined by a predetermined equation. Thereby, the respective image information and point cloud information obtained from the image capturing sensor and the laser scanning sensor may be combined and fused, and the point cloud information may have color information, or the image information may contain three-dimensional position coordinates for each pixel. can have As described above, the point cloud information is converted into virtual image data, and this data may be employed as reference image data.

In addition, the reference image database unit 116 stores information related to a specific type of object, ie, object information, in association with the reference image data. For example, the object may correspond to a building that needs to be displayed on a map, a traffic sign that needs to be recognized for autonomous driving, a traffic light, a lane, and the like. In this case, the object information may include the type of the object, the overall shape, a specific geometric shape such as a line, a surface, a circle, or a sphere constituting at least a part of the object, a color, and a texture-related attribute.

On the other hand, the search selection unit 118 obtains CCTV geometric information in relation to the image data acquired from the CCTV device 102, and at least refers to the reference position data and the two-dimensional position data to search for image data that can be matched with the reference image data. to select The search selection unit 118 searches the two-dimensional coordinates of the reference position data when the two-dimensional position data has coordinates, and the two-dimensional position data is the name of the installed space of the CCTV device 102, which is rough data, not coordinates. If this is the case, the reference image database unit 116 searches for registered reference position data related to the name of the space.

The reference position data may be composed of two-dimensional position data or three-dimensional position data having higher accuracy than the two-dimensional position data of the first external geometric data. When the reference position data is the two-dimensional position data, the search selection unit 118 searches for the CCTV geometry information and the second-dimensional position data of the reference geometry information. When the reference position data is three-dimensional position data, two-dimensional position data of CCTV geometry information and two-dimensional position data of XY coordinates extracted from three-dimensional position data are searched.

Unlike the above embodiment, when the CCTV geometric information includes all of the second posture data and the second internal geometric data, the search selection unit 118 is the first including the reference position data, the two-dimensional position data, and the posture data. and the second external geometric data and the first and second internal geometric data to search for a candidate group of image data that can be matched with the reference image data, and select image data having the highest degree of matching with the reference image data from the candidate group.

Specifically, the search selection unit 118 searches for a candidate group of image data that can be contrasted with the reference image data based on the reference position data and the two-dimensional position data, and refers to the first and second posture data for reference image data among the candidate groups. It is possible to select the image data having the best degree of matching with .

When the image data obtained from the CCTV device 102 originates from the wide-area image sensor, the search selection unit 118 removes the distortion of the wide-area image and converts it into flat image data. This can be done before the data selection process.

The matching unit 120 mutually matches the selected image data and the reference image data. The matching may be implemented by a method by feature geometry extraction or by a method by a template characteristic specified in an image.

First, in the method by feature geometry extraction, the reference image data and the selected image data are matched based on a predetermined type of feature geometry common between the reference image data and the selected image data, and the geometry such as the collinear conditional expression is referred to by referring to the feature geometry. A plurality of selected image data can be combined through the model.

The feature geometry of a predetermined shape may be extracted based on a specific geometric shape of an object expressed in the selected image data and the reference image data estimated to be the same location. The object may be randomly selected as an object from which feature geometries such as points, lines, planes, and columns can be easily extracted from image data, such as road signs, objects of unusual shapes, building boundaries, and the like. Also, feature geometry extraction may be performed based on color change in image data. Even if the posture data between the selected image data and the reference image data is slightly different, the characteristic geometry adopts a predetermined shape with a high probability of being estimated as the same object, and may be, for example, a lane marked on a road surface, a road sign, a building outline, and the like.

Here, the image data may be used not only as matrix-type data having color information, but also as a descriptor having a characteristic that can represent a specific object. In the case of a descriptor having a characteristic representing a specific object, SIFT (Scale Invariant Feature Transform) may be used as an image comparison algorithm.

In order to calculate the descriptor according to the SIFT, the order of Scale-space extrema, Key-point localization & Filtering, Orientation allocation, and descriptor generation may be performed.

Specifically, if a boundary is simply detected in image data, many edges including noise are detected. First, a Gaussian filter is applied at various levels (adjusting the σ value) to blur the image data, and the image data When the difference between and the reference image data is calculated, a characteristic robust to noise is obtained (DoG: Difference of Gauassian). When the work on the current scale is completed, the same work is performed while changing the image size to be able to respond to the scale change to obtain extrema, a robust feature at different sizes.

Even though relatively robust extrema is extracted through DoG, it still contains a lot of unnecessary information, so only meaningful information is extracted through keypoint localization and filtering. Next, a histogram according to the direction of the gradient is calculated.

In the NХN gradient window, NХNХm-dimensional descriptors are generated in m directions, and the matching of images is determined by comparing these descriptors.

According to the template characteristic method, the matching unit 120 searches for the selected image data that is at least similar to a template specified in the reference image data, and matches the selected image data according to the characteristic correlation analysis between the templates. The template may be, for example, an object occupying a considerable area in image data such as a predetermined sign or signboard. In order to improve the matching speed, the characteristic correlation analysis may use a method of matching a similar characteristic region by converting a spatial characteristic of an image into a frequency characteristic through a Fourier transform. The feature geometry or template constitutes matching information, and the matching information may be stored in the reference image database unit 116 .

The geometric transformation unit 122 geometrically transforms the CCTV geometry information of the selected image data by geometric transformation modeling referring to reference image data in common feature geometries or templates for correction with higher accuracy than the two-dimensional position data Generate corrected CCTV geometry that includes location data.

For geometric transformation modeling, one of perspective transformation, affine transformation, and homography transformation, which can reflect the geometry of image data in a matrix form, may be used, or alternative modeling may be used.

[Equation 1]

The image coordinates are two-dimensional position data of image data related to the same feature geometry or template, and the object coordinates are three-dimensional position data of reference image data for the same feature geometry or template. The camera position and posture are first external geometric data of the reference image data, and the main point position, focal length, and lens distortion are first internal geometric data. In the above description, an embodiment in which the reference image data is expressed as 3D position data has been described by taking the case where the reference position data is 3D position data as an example, but when the reference position data is 2D position data, the reference image data is 2D It can be expressed as two-dimensional position data of coordinates.

The matching optimal value estimator 124 randomly samples the feature geometries of the image data for the sample to which the CCTV geometry information for correction is linked, and the amount of mutual separation between the feature geometries for the sampled sample and the feature geometries of the corresponding reference data is minimized. The feature geometry for the sample can be estimated as the matching optimal value.

Specifically, a plurality of image data sharing the same estimated feature geometry for a sample is sampled. A plurality of such sampling sets are acquired for different standard feature geometries. At the same time, reference image data having the same estimated feature geometry as the feature geometry for the sample is linked and selected. Then, in the case of arranging the image data and the reference image data that share the same estimated feature geometries for the sample according to the CCTV geometry information for correction, the mutual distance between the feature geometries for the sample of the image data and the feature geometry of the reference image data is The feature geometry for the sample of the image data judged to be the minimum is estimated as the matching optimal value.

The CCTV geometry information estimator 126 may estimate the geometry-converted correction CCTV geometry information in relation to the feature geometry estimated as the above-described optimal matching value as the final correction CCTV geometry information.

The image information generating unit 128 generates CCTV image information from the image data of the CCTV device 102 based on the final corrected CCTV geometry information and matching information for the feature geometry.

CCTV image information is, like the reference image data of the reference image database unit 116, corrected image data corrected by geometric transformation and position data associated therewith, corrected second external geometric data including corrected second posture data, It may include the final corrected CCTV geometry information, color data, time data, etc. including the corrected second internal geometric data.

Position data associated with the corrected image data may be composed of two-dimensional position coordinates when the reference position data is two-dimensional position data, and may be composed of three-dimensional position coordinates when the reference position data is three-dimensional position data.

On the other hand, the reference geometric information is composed of three-dimensional position data as reference position data and first external geometric data having first posture data, and the precision of the reference geometry information of the reference image data related to some positions of the image data is lower than the threshold value. In this case, the image information generating unit 128 may convert the three-dimensional image information of the image data of the CCTV device 102 corresponding to the low-precision reference image data into a two-dimensional geometric model.

The image information generating unit 128 analyzes the object height by estimating the object height through pixel data analysis within the two-dimensional converted image data, or by assuming the height of the image sensor 106 of the CCTV device 102 as a specific height. Alternatively, the second posture data related to the viewing direction of the multiple image sensors of the CCTV device 102 may be defined as a specific value, and the object height may be estimated using a trigonometric model. The 3D image information of the corresponding image data may be generated by estimating the 3D position data by reflecting the object height estimated by this to the final corrected CCTV geometry information. This may be applied when the reference image data has low precision of reference geometric information, such as an aerial image or a street view (street view).

The object information extraction unit 130 may extract a specific type of object from the CCTV image information resulting from the image data.

Specifically, the object information extraction unit 130 may extract object information related to the property of the extracted object based on the data included in the CCTV image information.

The object information extraction unit 130 detects the candidate group data related to the object identified from the image data included in the CCTV image information, the final corrected CCTV geometric information, color data, time data, etc. by machine learning technique, and By sequentially applying additional machine learning models, the information of the object can be recognized.

Machine learning is performed in two stages: target object detection and target object recognition. In the target object detection step, candidate groups are detected through machine learning based on properties such as vectors such as shape, color, and texture of the target object. Then, an additional machine learning model is sequentially applied to the detected candidate group to recognize and classify the information indicated by the target object.

The spatial information generating unit 132 is based on the object of the same reference image data as the extracted object, the final corrected CCTV geometry information and a predetermined coordinate system including object information related to the object stored in the reference image data in the CCTV image information, for example, absolute New spatial information according to the coordinate system may be generated and transmitted to and stored in the spatial information database unit 136 . Spatial information is spatial data to be described later, and detailed information will be described later.

The spatial information database unit 136 structures and stores observation data previously acquired from heterogeneous sensors and data previously generated for drawing as spatial data according to a predetermined coordinate system, for example, an absolute coordinate system. Spatial data is data generated in corresponding coordinates of a predetermined coordinate system, and is observed data, internal geometry, external geometry, intrinsic data, time information, acquisition environment information, separation amount due to position error of sensor data between heterogeneous sensors, accuracy, and precision It may include related probability information, point cloud related information, image data related information, object information, and map related information. In the present embodiment, it is described that spatial data includes all of the above-described data or information, but is not limited thereto, and may be set to include only some of the above-described data/information according to design specifications.

When the existing spatial information already stored in the spatial information database unit 136 exists, the spatial information update unit 134 determines whether new spatial information is different from the existing spatial information in object information of the same location; As a result of the determination, for different object information, existing spatial information may be updated with new spatial information.

The ground surface spatial information extraction unit 138 uses CCTV image information generated from the image data of the CCTV device 102 based on the CCTV geometric information and matching information for correction, the CCTV in the Region of Interest (ROI) The earth surface spatial information about the earth surface included in the image data of the device 102 and an object adjacent to the earth surface is extracted. CCTV image information may be generated from image data of the CCTV device 102 by the image information generating unit 128 that generates three-dimensional spatial information (including information formed from estimated three-dimensional position data). As shown in FIG. 18 , the region of interest may be selected to include an object having a shape formed vertically from the ground surface, and the ground surface spatial information may include 3D spatial information along a vertical direction of the object. The surface spatial information may be stored in the spatial information database unit 136 and used as a reference value when measuring the level of inundation in the area of interest when a flood occurs. The 3D spatial information of the ground surface spatial information may be set to conform to a predetermined coordinate system of the spatial information database unit 136 . The surface spatial information may be designated as spatial information of an object pre-stored in the spatial information database unit 136 , and the spatial information of the designated object may be used as a reference value when measuring the inundation level of an ROI related to the object.

The submersion level estimator 140 estimates the submergence level of the region of interest by comparing the spatial information of the object included in the image data of the region of interest obtained in real time when a flood occurs, with the spatial information on the ground. As shown in FIG. 18 , the submergence level estimation unit 140 estimates the flood level in real time by the difference between the 3D spatial information of the object exposed at the same level as the water level due to the flood and the 3D spatial information of the surface spatial information. can In another embodiment, the submersion level estimator 140 analyzes the image based on the difference between the previously registered normal object image and the object image at the time of the occurrence of a flood through surface spatial information, and estimates the height of the water surface according to the difference between the images. can be carried out.

According to this embodiment, the reference image information of the reference image data generated from the multi-sensor platform consisting of the observation sensor and the positioning sensor for the image data of the CCTV device 102 having geometric data related to the position and posture that are inaccurate or missing in some areas By matching with and performing geometric transformation correction, etc., it is possible to generate image data acquired from a CCTV device as spatial information with convenience and high precision. Specifically, when the two-dimensional position data associated with the video data of the CCTV device 102 has low accuracy, or is omitted in the first place, or is recorded as a rough spatial name, the CCTV image corrected by the reference geometric information The information may have position data composed of coordinates with higher accuracy than the original image data.

In particular, by linking the CCTV image of the flood-damaged area obtained from the CCTV device in which geometric information such as location and posture has been converted with the usual spatial information of the flood-damaged area, from the CCTV image of the flood-damaged area obtained in real time when a flood occurs, the The flood level can be accurately estimated. In addition, by detecting and tracking changes in the flood level in real time and linking with the disaster response plan, it is possible to update and provide more accurate flood level information compared to the previous one, and by including information on the trend of changes in the flood level over time, A systematic disaster response can be achieved.

In addition, it is possible to link and analyze the corrected position and posture information and the image information obtained from CCTV to spatialize the object information expressed in the image and link it with the map service.

In addition, by processing the observation data obtained from low-cost sensors excluding expensive sensors such as lidar when constructing the reference image data, it is possible to achieve reduction in construction cost.

In addition, in this embodiment, the final corrected CCTV geometry information, etc. is generated with the feature geometries with the minimum amount of mutual separation between the feature geometries estimated to be the same through the matching optimal value estimator 124, but the second external geometric data is In the case of data capable of achieving good matching between image data, the matching optimal value estimator 124 may be omitted if the condition that the mutual spacing between the feature geometries is less than or equal to a threshold value is satisfied.

Hereinafter, with reference to FIGS. 2 to 18, FIG. 2 is a process of generating and estimating the flood level of CCTV image information for the video data of the CCTV device implemented in the CCTV device-based flood level estimation system according to an embodiment of the present invention. to explain about In this embodiment, the function of the matching optimal value estimator 124 is performed. However, as described above, this may be omitted.

2 is a flowchart of a method for generating CCTV image information for image data of a CCTV device and for estimating a submersion level implemented in a CCTV device-based flood level estimation system according to an embodiment of the present invention.

First, the search selection unit 118 acquires CCTV geometric information including the second external geometric data composed of two-dimensional position data related to the image data acquired from the CCTV device 102 (S205).

Next, the search selection unit 118 performs at least the reference position data and the image data of the reference geometry information of the reference image data including the first external geometric data and the first internal geometric data having the reference position data and the first posture data. A candidate group of image data is searched and selected based on the two-dimensional position data (S210).

The two-dimensional location data may be two-dimensional coordinates of the CCTV device 102, or the name of a space in which the CCTV device is installed as rough information rather than coordinates. If the two-dimensional position data is searched for two-dimensional coordinates of three-dimensional position data when the two-dimensional position data has coordinates, and the two-dimensional position data is the name of the installed space of the CCTV device 102, which is rough data, not coordinates, the reference image database unit In step 116, the registered three-dimensional position data related to the name of the space is retrieved.

The reference position data may be composed of two-dimensional position data or three-dimensional position data having higher accuracy than the two-dimensional position data of the first external geometric data. When the reference position data is the two-dimensional position data, the search selection unit 118 searches for the CCTV geometry information and the second-dimensional position data of the reference geometry information. When the reference position data is three-dimensional position data, two-dimensional position data of CCTV geometry information and two-dimensional position data of XY coordinates extracted from three-dimensional position data are searched.

The reference image data is stored in the reference image database unit 116, and in the case of the reference image data linked to the 3D position data, as shown in FIG. is saved The three-dimensional position data may be two-dimensional position data, ie, XYZ coordinates obtained by adding an altitude value Z to the XY coordinates.

As such, the first external geometric data includes the above-described first posture data together with the reference position data, and the first internal geometric data is based on a geometric parameter defined in a sensor configured to generate a reference image.

The reference image is an image that is obtained from the outside and includes at least reference position data, and is processed into reference image data that can be contrasted with the image data obtained from the CCTV device 102 . The sensor initially used to generate the reference image may be at least one of an image acquisition device and a laser scanning sensor (eg, Lidar) having higher positioning accuracy and image realization performance than the CCTV device 102 . The reference geometric information associated with the reference image includes color information such as RGB data, first external geometric data, and first internal geometric data for each pixel coordinate, and the reference image combined with the reference geometric information is stored as reference image data can be

The reference image data matches and corrects image information and point cloud information obtained from an external mobile platform having at least one of an image capturing sensor (image acquisition device) and a laser scanning sensor and a positioning sensor through a geometric model between the sensors. This may be external image data fused by the fusion, or image data of an existing CCTV device that has been corrected with corrected CCTV geometry information and is already stored in the image information generating unit 128 and/or spatial information database unit 136 as image information.

When the sensor initially used to generate the reference image is an image capturing sensor, the first internal geometric data is substantially the same as the image sensor 106 described in the second internal geometric data. In the case of an image capturing sensor, as shown in FIG. 4, the projection center and direction used to project the three-dimensional shape of the object in two dimensions are related to the three-dimensional position data and the viewing direction of the reference image, so the projection center and direction are 1 It may be stored as reference image data together with external geometric data. The reference geometry information of the reference image data generated by the image capturing sensor may include two-dimensional position data or three-dimensional position data as reference position data.

When the reference image is aerial observation data, since it has a different geometry from ground observation image data such as the CCTV device 102, the aerial observation data is in the form of a view in a specific direction, for example, in a top view shape, as shown in FIG. 5 . can be converted For example, each pixel of the aerial observation data may be converted into an orthogonal image form having a horizontal geometry and employed as a reference image.

When the reference image originates from a wide-area image sensor that generates a distorted image, the reference image uses planar image data generated by converting the distorted image into planar image data, and the reference position data uses position data of the planar image data. and the first posture data may be defined as a viewing direction, a viewing angle, etc. of the planar image data.

In this regard, a process of removing distortion and generating planar image data performed in the reference image database unit 116 will be described with reference to FIGS. 6 to 10 .

6 is a flowchart of a first process of converting image data acquired from a wide-area image sensor into flat image data by removing distortion, and FIGS. 7 and 8 are schematic views of a first process of image distortion removal and plane image data conversion. It is the drawing shown.

9 is a flowchart of a second process of converting image data acquired from a wide-area image sensor to flat image data by removing distortion, and FIG. 10 is a diagram schematically illustrating a second process of image distortion removal and plane image data conversion. am.

Referring to FIG. 6 related to the first process, when image data for a fisheye lens distorted due to a wide-area image sensor equipped with a fisheye lens is input (S605), the reference image database unit 116 is configured as shown in FIG. , an index is assigned to each pixel to define a correspondence between the distorted image data from the image data and the plane image data for reference ( S610 ).

Next, the reference image database unit 116 decomposes the indexed distorted image data into preset sections (S615).

Next, the reference image database unit 116 generates flat image data by removing the distortion of the distorted image data based on the distortion correction model of the decomposed section with reference to the flat image data for reference ( S620 ).

The distortion correction model may be modeling based on a formula or an index of the decomposed section.

If the omnidirectional image data is input, the reference image database unit 116 decomposes the omnidirectional image data into preset sections as in the second step of FIG. 8 .

Next, the reference image database unit 116 re-decomposes the decomposed omnidirectional image into a preset section, as in the third step of FIG. 8 . Next, the reference image database unit 116 removes the distortion based on a distortion correction model of the re-decomposed section, for example, an equation or an index, and generates flat image data as shown in the right image of FIG. 8 .

This is substantially the same process even when the image data received from the CCTV device 102 includes a distorted image due to the wide area image sensor, and this process can be performed in the search selection unit 118 for inputting the image data. there is.

On the other hand, referring to FIG. 9 related to the second process, when omniazimuth image data distorted due to a wide area image sensor generating an omnidirectional image is input to the reference image database unit 116 (S905), the reference image database unit ( In step 116, a virtual 3D shape is restored from the distorted image data based on the camera geometry based on the distorted image data from the image data (S910).

Next, the reference image database unit 116 generates plane image data by defining a virtual camera geometry and projecting it to the virtual plane camera through the geometry (S915).

Alternatively, when the reference image originates from the laser scanning sensor, the reference image may use point cloud-converted virtual image data obtained from the laser scanning sensor. The reference position data is 3 of the virtual image data. It is defined as dimensional position data, and the first posture data may be defined as a viewing direction of the virtual image data.

Transformation of the point cloud may be performed by virtual camera modeling in consideration of the reflection intensity and elevation value of the point cloud located below the drawing as shown in FIG. 11 . As a result, virtual image data may be generated as shown in the example of FIG. 11 . When the image sensor is additionally involved in generating the reference image, color data obtained by the image sensor when the point cloud is converted may be included in the virtual image data.

In step S210, the search selection unit 118 selects the image data by referring only to the reference position data and the two-dimensional position data, but when the CCTV geometry information includes all of the second posture data and the second internal geometry data, the search line The government 118 may proceed with the process of FIG. 12 . 12 is a flowchart related to another embodiment of search selection in a method of generating spatial information for image data of a CCTV device.

First, the search selection unit 118 is related to the image data acquired from the CCTV device 102, the CCTV geometric information including the second external geometric data and the second internal geometric data having two-dimensional position data and second posture data Acquire (S1205)

The second external geometric data is based on a parameter value that is changed every time the image information of the image sensor 106 is acquired due to the position and posture of the CCTV device 102 in motion, that is, the position and posture of the image sensor 106 . This is the error of the observed data calculated by doing this, and in this case, it may be calculated through a predetermined formula or modeling. The second internal geometric data is an intrinsic value of the sensors 104 to 108 itself, and is an error of observation data for each sensor 104 to 108 due to a parameter maintained regardless of whether the CCTV device 102 moves. The geometric parameters for each of the sensors 104 to 108 are omitted above.

Next, the search selection unit 118 performs at least the reference position data and the image data of the reference geometry information of the reference image data including the first external geometric data and the first internal geometric data having the reference position data and the first posture data. A candidate group of image data is searched based on the two-dimensional position data (S1210).

Here, the image data may be used not only as matrix-type data having color information, but also as a descriptor having a characteristic that can represent a specific object. In the case of a descriptor having a characteristic representing a specific object, SIFT (Scale Invariant Feature Transform) may be used as an image comparison algorithm. This will be described in detail in the description of the system 100 and will be omitted.

Next, the search selection unit 118 selects image data having the highest degree of matching with the reference image data from the candidate group with reference to the two-dimensional position data, the three-dimensional position data, and the first and second posture data (S1215).

Specifically, if the first and second posture data of the first and second external geometric data are different according to the viewing direction and viewing angle of the image data having the same or very similar two-dimensional position data as the reference image data, the image geometry Even the same feature geometry is recognized as different geometry, so it is difficult to match the reference image data and the image data. Accordingly, by selecting only image data similar to the first and second posture data within a predetermined range at the same position from the candidate group, the accuracy and processing speed of the subsequent matching step may be improved.

Referring back to FIG. 2 , the matching unit 120 matches the selected image data with the reference image data ( S215 ).

The matching can be implemented by a method by feature geometry extraction according to FIG. 13 or by a method by a template characteristic specified in the image according to FIG. 14 .

13 is a diagram illustrating a matching process using feature geometry in the matching unit, and FIG. 14 is a flowchart illustrating the matching process using characteristic correlation analysis between templates in the matching unit.

The method by extracting feature geometry as shown in FIG. 13 matches the reference image data and the selected image data based on predetermined types of feature geometries common between the reference image data and the selected image data, and refers to the feature geometry, such as a collinear condition equation, etc. It is possible to combine a plurality of selected image data through the geometric model of .

Here, the image data may be used not only as matrix-type data having color information, but also as a descriptor having a characteristic that can represent a specific object. In the case of a descriptor having a characteristic representative of a specific object, SIFT may be used as an image comparison algorithm.

Referring to FIG. 14 for the method based on the template characteristic, the matching unit 120 searches for the selected image data that is at least similar to a template specified in the reference image data (S1405).

Next, matching is performed according to the characteristic correlation analysis between the templates of the reference image data and the selected image data (S1410). The template may be, for example, an object occupying a considerable area in image data such as a predetermined sign or signboard. In order to improve the matching speed, the characteristic correlation analysis may use a method of matching a similar characteristic region by converting a spatial characteristic of an image into a frequency characteristic through a Fourier transform.

Next, the reference image data and the selected image data are matched with templates having a characteristic correlation (S1415).

The feature geometry described with reference to FIG. 13 or the template described with reference to FIG. 14 constitutes matching information, and the matching information may be stored in the reference image database unit 116 .

Referring back to FIG. 2 , the geometric transformation unit 122 geometrically transforms CCTV geometry information of selected image data by geometric transformation modeling referring to reference image data in common feature geometries or templates to perform two-dimensional position data Creates a corrected CCTV geometry information that contains more accurate position data for correction (S220).

For the geometric transformation modeling, one of perspective transformation, affine transformation, and homography transformation that can reflect the geometry of image data in a matrix form may be used, or alternative modeling may be used.

15 is a diagram illustrating a process of calculating two-dimensional position data of image data and reference position data of reference image data corresponding to the two-dimensional position data of image data. In the case of using the collinear conditional expression, image data is geometrically transformed into three-dimensional position coordinates of the reference image data and corrected in the same feature geometry.

Next, the matching optimal value estimator 124 randomly samples the feature geometries of the image data to which the CCTV geometry information for correction is linked, and the sampled feature geometries and the sampled feature geometries of the corresponding reference data are the minimum sampling The feature geometry of the image data is estimated as a matching optimal value (S225). The detailed process of the matching optimal value will be omitted as described above.

Next, the CCTV geometry information estimator 126 estimates the geometrically transformed three-dimensional CCTV geometry information in relation to the feature geometry estimated as the above-described optimal matching value as the final corrected CCTV geometry information (S230).

Next, the image information generating unit 128 generates CCTV image information for the image data of the CCTV device 102 based on the final corrected CCTV geometry information and matching information for the feature geometry (S235).

CCTV image information is, like the reference image data of the reference image database unit 116, corrected image data corrected by geometric transformation and position data associated therewith, corrected second external geometric data including corrected second posture data, It may include the final three-dimensional CCTV geometric information, color data, time data, etc. including the corrected second internal geometric data.

The position data associated with the corrected image data may consist of two-dimensional position coordinates when the reference position data is two-dimensional position data, and may consist of three-dimensional position coordinates when the reference position data is three-dimensional position data. .

16 and 17, when the reference geometric information is composed of three-dimensional one-dimensional position data as reference position data and first external geometric data having first posture data, and the precision of the reference geometry information included in the reference image data is low An object height estimation process performed by the image information generator 128 will be described.

16 is a flowchart illustrating a process of estimating the height of an object of CCTV image data in the 3D image information generator, and FIG. 17 is a diagram illustrating a process of estimating the height of an object through pixel data analysis in CCTV image data .

The process of FIG. 16 is performed when the precision of the reference geometric information related to a partial position of the reference image data having the 3D position data is lower than a threshold value.

First, the image information generating unit 128 converts the three-dimensional image information of the image data of the CCTV device 102 corresponding to the low-precision reference image data into a two-dimensional geometric model (S1605).

Next, the image information generating unit 128 estimates the object height through pixel data analysis within the two-dimensional converted image data, or analyzes the object height by assuming that the height of the image sensor of the CCTV device is a specific height, or CCTV The second posture data related to the viewing directions of the plurality of image sensors of the device is defined as a specific value, and the object height is estimated using a trigonometric model ( S1610 ).

In the case of pixel data analysis, as in FIG. 17 , pixels in the image data are analyzed for each horizontal and vertical pixel to determine the height of the image sensor 106 , the angle h between the horizontal line from the image sensor 106 and the highest point of the object, and the image sensor The angle (a) between the horizontal line and the extension line from the central focus (f) of (106) is calculated, and the height of the object 202 is estimated based on this in consideration of the number of pixels and the vanishing point. Accordingly, 3D image information is collected.

Subsequently, the image information generating unit 128 reflects the estimated object height in the three-dimensional CCTV geometric information to estimate the three-dimensional position data, thereby generating the three-dimensional image information of the image data of the corresponding CCTV device 102 . do (S1615).

Referring back to FIG. 2, after step S235, the ground surface spatial information extraction unit 138 uses the CCTV image information generated from the image data of the CCTV device 102 based on the CCTV geometry information and matching information for correction, In the region of interest (ROI), the spatial information of the ground surface included in the image data of the CCTV device 102 and the object adjacent to the ground surface is extracted (S240). The CCTV image information may be generated from the image data of the CCTV device 102 by the image information generating unit 128 that generates three-dimensional spatial information.

The region of interest may be selected to include an object (refer to the box of FIG. 18 ) having a shape formed vertically from the ground surface as in FIG. 18 , and the surface spatial information may include 3D spatial information along the vertical direction of the object. can The surface spatial information may be stored in the spatial information database unit 136 and used as a reference value when measuring the level of inundation in the area of interest when a flood occurs. The surface spatial information may be designated as spatial information of an object pre-stored in the spatial information database unit 136 , and the spatial information of the designated object may be used as a reference value when measuring the inundation level of an ROI related to the object. The 3D spatial information of the earth's spatial information may be set to conform to a predetermined coordinate system of the spatial information database unit 136 .

Next, the submersion level estimating unit 140 compares the spatial information of the object included in the image data of the region of interest acquired by the CCTV device 102 in real time when a flood occurs, against the spatial information on the ground, to determine the submersion level of the region of interest. Estimate (S245).

As shown in FIG. 18 , the submersion level estimation unit 140 is a three-dimensional space of the object exposed at the same level as the water level due to the water damage (the exposed portion of the vertical column object exposed on the same level as the water surface of FIG. 18 ). The inundation level can be estimated in real time by the difference between the information and the three-dimensional spatial information of the surface spatial information. In another embodiment, the submersion level estimator 140 analyzes the image based on the difference between the normal object image and the object image at the time of flood occurrence, which are previously registered through the ground surface spatial information, and estimates the height of the water surface according to the difference between the images. can be carried out. In this case, when there is no 3D spatial information for each point of the object, the submergence level may be estimated by estimating the height of the water surface through image analysis.

According to this embodiment, three-dimensional image information of the reference image data generated from the multi-sensor platform composed of the observation sensor and the positioning sensor for the image data of the CCTV device 102 having inaccurate or missing position and posture-related geometric data in some areas By matching with and performing geometric transformation correction, etc., it is possible to generate the image data acquired from the CCTV device 102 as 3D spatial information with convenience and high precision.

In particular, by linking the CCTV image of the flood-damaged area obtained from the CCTV device in which geometric information such as location and posture has been converted with the usual spatial information of the flood-damaged area, from the CCTV image of the flood-damaged area obtained in real time when a flood occurs, the The flood level can be accurately estimated. In addition, by detecting and tracking changes in the flood level in real time and linking with the disaster response plan, it is possible to update and provide more accurate flood level information compared to the previous one, and by including information on the trend of changes in the flood level over time, A systematic disaster response can be achieved.

In addition, by processing the observation data obtained from low-cost sensors excluding expensive sensors such as lidar when constructing the reference image data, it is possible to achieve reduction in construction cost.

Hereinafter, a method of generating and updating spatial information implemented in a CCTV device-based flood level estimation system will be described with reference to FIGS. 19 to 21 .

19 is a flowchart of a method for generating and updating spatial information implemented in a CCTV device-based flood level estimation system.

20 is a diagram illustrating an example of generating new spatial information including final corrected CCTV geometry information and object information in CCTV image information.

21 is a diagram for explaining a process of updating spatial information.

First, the object information extraction unit 130 extracts a specific type of object from the CCTV image information resulting from the image data (S1905).

Specifically, the object information extraction unit 130 may extract object information related to the property of the extracted object based on the data included in the CCTV image information.

The object information extraction unit 130 detects the candidate group data related to the object identified from the image data included in the CCTV image information, the final corrected CCTV geometric information, color data, time data, etc. by machine learning technique, and By sequentially applying additional machine learning models, the information of the object can be recognized.

Next, the spatial information generating unit 132, based on the object of the same reference image data as the extracted object, the final corrected CCTV geometry information and a predetermined coordinate system including object information related to the object stored in the reference image data in the CCTV image information , for example, new spatial information according to the absolute coordinate system is generated (S1910).

The spatial information generating unit 132 may transmit and store new spatial information to the spatial information database unit 136 . Spatial information is data generated in corresponding coordinates of a predetermined coordinate system, and is observed data, internal geometry, external geometry, intrinsic data, time information, acquisition environment information, separation amount due to position error of sensor data between heterogeneous sensors, accuracy, and precision It may include related probability information, point cloud related information, image data related information, object information, and map related information. In the present embodiment, although spatial information is described as including all of the above-described data or information, the present invention is not limited thereto, and may be set to include only some of the above-described data/information according to design specifications.

As shown in FIG. 20, new spatial information is generated by including the final corrected CCTV geometry information and object information in CCTV image information, and based on this, the map on the right can be produced through a drawing process.

Next, when the existing spatial information already stored in the spatial information database unit 136 exists, the spatial information update unit 134 determines whether the new spatial information is different from the existing spatial information in object information of the same location. It is determined (S1915), and as a result of the determination, for different object information, the existing spatial information is updated with new spatial information as shown in FIG. 20 (S1925). As a result of the determination, when the object information is the same, the existing spatial information may be maintained (S1920).

Components constituting the apparatus shown in FIG. 1 or steps according to the embodiments shown in FIGS. 2, 6, 9, 12, 14, 16, 19 are computer-readable recording media in the form of programs realizing the functions can be recorded in Here, the computer-readable recording medium refers to a computer-readable recording medium in which information such as data or a program is stored by electrical, magnetic, optical, mechanical, or chemical action. Among these recording media, for example, portable storage, flexible disk, magneto-optical disk, CD-ROM, CD-R/W, DVD, DAT, memory card, etc. are available as those that can be separated from the computer. In addition, as a recording medium fixed to a CCTV device and a computer, there is a solid state disk (SSD), a hard disk, or a ROM.

In addition, even though it has been described above that all components constituting the embodiment of the present invention operate by being combined into one, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more. In addition, all of the components may be implemented as one independent hardware, but some or all of the components are selectively combined to form a program module that performs some or all of the functions combined in one or a plurality of hardware. It may be implemented as a computer program having

Although the present invention has been described in detail through representative embodiments above, those of ordinary skill in the art to which the present invention pertains can make various modifications to the above-described embodiments without departing from the scope of the present invention. will understand Therefore, the scope of the present invention should not be limited to the described embodiments and should be defined by all changes or modifications derived from the claims and equivalent concepts as well as the claims to be described later.

Claims (11)

a reference image database unit for storing reference image data in association with reference position data and reference geometry information including first external geometry data having first posture data defining a viewing direction set to generate a reference image;
Obtain CCTV geometric information including second external geometric data having at least two-dimensional location data in relation to the video data acquired from the CCTV device, and at least the reference video data and the reference video data with reference to the reference location data and the two-dimensional location data a search selection unit that searches for and selects the matching image data;
a matching unit for mutually matching the reference image data and the selected image data;
Geometric transformation unit that geometrically transforms the CCTV geometry information of the selected image data by geometric transformation modeling referring to the reference image data to generate corrected CCTV geometry information that includes position data for correction with higher accuracy than the two-dimensional position data ;
By using the CCTV image information generated from the image data of the CCTV device based on the CCTV geometry information and matching information for correction in the image information generating unit, the video data of the CCTV device in a Region of Interest (ROI) a ground surface spatial information extraction unit for extracting ground surface spatial information on the ground surface included in the , and an object adjacent to the ground surface; and
and a flood level estimator for estimating a flood level from the image data of the region of interest when a flood occurs based on the surface spatial information.
The method of claim 1,
The region of interest is selected to include an object having a shape formed vertically from the ground surface, and the ground surface spatial information includes spatial information along a vertical direction of the object, CCTV device-based inundation level estimation system.
The method of claim 1,
The submergence level estimator estimates the submergence level of the area of interest by comparing the spatial information of the object included in the image data of the area of interest obtained in real time when a flood occurs, against the spatial information on the ground, but is the same as the water level due to the water damage The inundation level is estimated in real time by the difference between the three-dimensional spatial information of the object exposed at the level and the three-dimensional spatial information of the surface spatial information, or the image of the object of the region of interest in normal times related to the surface spatial information and the occurrence of flood damage A CCTV submersion level estimation system for estimating the submersion level in real time by analyzing the difference between object images in the region of interest.
The method of claim 1,
The two-dimensional position data includes two-dimensional coordinates of the CCTV device or the name of a space in which the CCTV device is installed, and the reference position data is two-dimensional position data with higher accuracy than the two-dimensional position data of the first external geometric data. or three-dimensional position data, and when the reference position data is two-dimensional position data, the CCTV image information includes two-dimensional position coordinates, and when the reference position data is three-dimensional position data, the CCTV Video information, including three-dimensional position coordinates, CCTV device-based submergence level estimation system.
The method of claim 1,
The second external geometric data further includes second posture data defining a viewing direction set to generate the image,
The search selector searches for a candidate group of the image data that can be contrasted with the reference image data based on the reference position data and the two-dimensional position data, and refers to the first and second posture data for the reference image among the candidate group. A CCTV device-based flood level estimation system that selects the video data with the highest degree of matching with the data.
The method of claim 1,
The reference image data is fused by matching and correcting image information and point cloud information obtained from an external mobile platform having at least one of an image capturing sensor and a laser scanning sensor and a positioning sensor through a geometric model between the sensors. A CCTV device-based flood level estimation system, which is corrected image data already stored as CCTV image information after being corrected with external image data or the CCTV geometry information for correction.
The method of claim 1,
When at least one of the reference image and the image data is obtained from a wide-area image sensor generating a distorted image, the reference image and the image data from which the distorted image is generated are a plane in which the distorted image is converted into a flat image. Using image data, and when the reference image uses the plane image data, the first posture data is defined as a viewing direction of the plane image data, a CCTV device-based flood level estimation system using reference image information.
The method of claim 1,
When the reference image originates from the laser scanning sensor, the reference image uses the point cloud-converted virtual image data obtained from the laser scanning sensor, and the reference position data is of the virtual image data. It is defined as position data, and the first posture data is defined as a viewing direction of the virtual image data, a CCTV device-based flood level estimation system.
The method of claim 1,
The matching unit extracts and compares feature geometries from the selected image data and the reference image data to match commonly estimated feature geometries, or searches for the selected image data that is at least similar to a template specified in the reference image data Thus, matching is performed according to the characteristic correlation analysis between the templates, the matching information is the characteristic geometry or the template, and the singi characteristic geometry or the template is stored in the reference image database unit, based on CCTV device using reference image information of the flood level estimation system.
The method of claim 1,
By randomly sampling the sample feature geometries of the video data to which the CCTV geometry information for correction is linked, the feature geometry for which the separation amount between the sample feature geometries and the corresponding reference data feature geometries is minimized as a matching optimal value an optimal value estimator for estimating; and
Further comprising a geometry information estimating unit for estimating the geometrical transformed CCTV geometry information for correction in relation to the feature geometry estimated as the matching optimal value as the final corrected CCTV geometry information,
Among the information based on the generation of the CCTV image information, the correction CCTV geometric information uses the final corrected CCTV geometry information, a CCTV device-based flood level estimation system.
The method of claim 1,
The reference geometry information is composed of three-dimensional position data as the reference position data and the first external geometry data having the first posture data, and the precision of the reference geometry information is a threshold value in relation to some positions of the reference image data. If lower than
The image information generator converts the corrected image data of the CCTV image information corresponding to the low-precision reference image data into a two-dimensional geometric model, and then calculates the object height through pixel data analysis in the two-dimensional converted image data. Estimating, analyzing the object height assuming the height of the image sensor of the CCTV device as a specific height, or defining the posture data related to the viewing direction of the multiple image sensors of the CCTV device as a specific value and using a trigonometric model, A CCTV device-based flood level estimation system for estimating an object height and estimating three-dimensional location data by reflecting the estimated object height to the corrected CCTV geometric information to generate the corresponding CCTV image information.

Images