WO2016125946A1

WO2016125946A1 - Panorama image monitoring system using plurality of high-resolution cameras, and method therefor

Info

Publication number: WO2016125946A1
Application number: PCT/KR2015/002035
Authority: WO
Inventors: 장정훈; 이형주
Original assignee: 주식회사 일리시스
Priority date: 2015-02-02
Filing date: 2015-03-03
Publication date: 2016-08-11
Also published as: KR20160094655A

Abstract

An image monitoring method that is executed in an image monitoring apparatus connected to a plurality of high-resolution cameras, according to an embodiment of the present invention, comprises the steps of: receiving high-resolution camera images from the plurality of high-resolution cameras; generating and displaying a low-resolution panorama image based on a low-resolution panorama image generation table using the high-resolution camera images; and generating and displaying a high-resolution digital PTZ image corresponding to a marked area, using an ultra-high-resolution panorama image generation table when marking a particular location or a particular moving object in the low-resolution panorama image.

Description

[Specification】

[Name of invention]

Panoramic video surveillance system using a plurality of high resolution cameras and method thereof [Technical Field]

Embodiments of the present invention relate to a method for monitoring a panoramic image using a plurality of high resolution cameras and an apparatus for executing the same.

Background technology

<2> In general, a video surveillance system is a system that monitors a specific location by installing a camera at a specific location and displaying images from a camera installed at a specific location in another location.

<3> Among the video surveillance systems, the panoramic video-based surveillance system provides 180-degree or 360-degree wide-angle video, which has the advantage of being able to monitor a large area at once. In addition, when auto-tracking moving objects from video images for automating video surveillance, panoramic video has the advantage of allowing seamless tracking of moving objects in large areas.

<4> In general, a method of acquiring a panoramic video image includes attaching a fisheye lens or a reflector to a single high-resolution camera to obtain a wide-angle image of 180 degrees / 360 degrees, and images obtained from a plurality of radial cameras. Attached to each other-can be divided into a way to obtain a wide-angle panoramic image.

Recently, a "single camera + fisheye lens" method capable of acquiring a panoramic image with a relatively simple configuration is mainly used in a video surveillance system. However, this method has a problem in that the overall resolution of the panoramic image is determined by the resolution of a single camera, and the image distortion becomes more severe toward the edge of the lens. As a result, this method is mainly used for near field surveillance.

<6> On the other hand, the method of acquiring a panoramic image using a plurality of general cameras is more complicated than the "single camera + fisheye lens" method, and the method of generating a panoramic image is more complicated. Because it can overcome the problem of the method, it can be used not only for near field but also for remote video surveillance.

In general, when the angle of view of the camera is fixed, the higher the resolution of the camera image, the more detailed information can be obtained about the distant subject. Therefore, high resolution cameras are often used in wide area / remote surveillance systems.

<8> Operation of Individual Cameras in Multiple General Camera-Based Panoramic Video Surveillance Systems ^{• If the} image resolution is high, the panorama image created by pasting images from individual cameras has a very high image resolution. For example, if you use five normal cameras arranged horizontally, and the individual cameras have Fu 1 1 -HD (1920x1080) resolution, the maximum resolution of the panoramic image obtained by this is about 8448x1080. Assuming 15% overlap between camera images). Real-time processing is very difficult because the ultra-high resolution panoramic image requires huge CPU resources in real time (ie at least 15 F S or more) in real time without a dedicated hardware device. Therefore, the panoramic video surveillance system based on a plurality of high resolution cameras is difficult to operate in the same way.

[Detailed Description of the Invention]

[Technical Challenges]

In the present invention, a panoramic video surveillance system using a plurality of high resolution cameras is used to generate a low resolution panoramic image instead of generating a high resolution panoramic image for displaying and analyzing a panoramic image, and to resolve a subject of interest. In order to build an efficient high resolution panoramic video surveillance system by creating and using only the video (high resolution digital PTZ video) corresponding to the region of interest instead of generating the entire super high resolution panoramic video for the high resolution video table, It is an object to provide a device for performing this.

The task to be solved by the present invention is not limited to the task (s) mentioned above, and other task (s) not mentioned will be clearly understood by those skilled in the art from the following description. ᅳ

[Technical Solution]

According to embodiments of the present disclosure, an image monitoring method executed in a video surveillance apparatus connected to a plurality of high resolution cameras disposed in a radial form and part of an image overlapping each other may include receiving a high resolution camera image from the plurality of high resolution cameras. Generating, displaying, and displaying a low resolution panoramic image using the high resolution camera image based on the low resolution panoramic image generation table; and setting a super high resolution panoramic image generation table when a specific position or a specific moving object is designated in the low resolution panoramic image And generating and displaying a high resolution digital PTZ image corresponding to a specified area by using the same. ^'

Among the embodiments, the video monitoring apparatus connected to a plurality of high resolution cameras disposed in a radial form and having a portion of an image overlapping each other may include the plurality of high resolution cameras. An image input unit for receiving a high resolution camera image from a low resolution panoramic image generating unit for generating a low resolution panoramic image using the high resolution camera image based on a low resolution panoramic image generation table, or a specific position or a specific movement in the low resolution panoramic image If the object is designated, the high resolution digital PTZ image generating unit generates a high resolution digital PTZ image corresponding to the designated area by using the ultra high resolution panorama image generation table.

Specific details of other embodiments are included in the detailed description and the accompanying drawings.

Advantages and / or features of the present invention and methods for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and only the embodiments are to make the disclosure of the present invention complete, and in the technical field to which the present invention pertains. It is provided to fully alert those skilled in the art to the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

[Effects of the Invention】

According to the present invention, a low resolution panoramic image is generated from a high resolution image received from a plurality of high resolution image photographing devices arranged in a radial form to display a panoramic image, and when an event occurs through real time image analysis, By providing digital PTZ video using the input high resolution video, it is possible to build a high resolution panoramic video surveillance system using less CPU resources.

[Brief Description of Drawings]

1 is an overall configuration diagram illustrating a panoramic video surveillance system according to an embodiment of the present invention.

2 is a block diagram illustrating a panoramic image monitoring apparatus according to an embodiment of the present invention.

3 is a flowchart illustrating an embodiment of a panoramic image monitoring method according to the present invention.

4 is a flowchart illustrating another embodiment of the panoramic image monitoring method according to the present invention.

5 is a reference diagram for describing a process of generating a low resolution panoramic image by the panoramic image monitoring apparatus according to an exemplary embodiment of the present invention. 6 is a reference diagram for explaining a process of generating a high resolution digital PTZ image by the panoramic image monitoring apparatus according to an embodiment of the present invention.

[Best form for implementation of the invention]

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<23>

Referring to FIG. 1, a high resolution panoramic image surveillance system includes a plurality of image capturing apparatus 100 and an image surveillance apparatus 200 arranged in a radial form.

The plurality of image capturing apparatuses 100 arranged in a radial form are provided with portions of the photographing regions overlapping each other, and provide a high resolution image to the image monitoring apparatus 200 by capturing a surrounding image.

Here, the plurality of image capturing devices 100 may be fixed in a radial form to obtain a high resolution image by capturing a corresponding area, and may be, for example, a high resolution megapixel camera. . -

When the video surveillance apparatus 200 receives the high resolution images from the plurality of video photographing apparatuses 100, the video surveillance apparatus 200 generates and displays the low resolution panoramic images based on the low resolution panoramic image generation table.

The video surveillance apparatus 200 generates and displays a low resolution panoramic image, performs object detection / tracking and event detection through real-time image analysis, and when a specified event is detected (for example, an “intrusion” event is generated). The moving object appears in the designated area of the panoramic image), and generates and displays a high-resolution digital PTZ image corresponding to the object or the event occurrence position.

As described above, the present invention generates and displays a low resolution panoramic image using a high resolution image received from a plurality of imaging apparatuses arranged in a radial form, and when an event occurs through real time image analysis, a high resolution of a corresponding position is provided. By providing a digital image, it is possible to effectively build and operate an ultra-high resolution panoramic video surveillance system using less CPU resources. Hereinafter, the video surveillance apparatus will be described in more detail with reference to FIG. 2.

2 is a block diagram illustrating a video surveillance apparatus according to an embodiment of the present invention. Referring to FIG. 2, the video surveillance apparatus 200 includes an image input unit 210 and an image decoding unit.

211, image storage unit 212, image retrieval / playback unit 213, low resolution panoramic image generation unit 214 ᅳ low resolution panoramic image generation table 215, object tracking unit 216, manual digital PTZ control unit ( 217, an automatic digital PTZ control unit 218, a high resolution digital PTZ image generating unit 219, an ultra-high resolution panoramic image generating table 220, and an image display unit 221.

The image input unit 210 receives a compressed high resolution image from a plurality of image capturing apparatuses 100 arranged in a radial shape and a portion of the photographing area overlaps each other, thereby receiving an image decoding unit 211 and an image storage unit. (212) provided to each.

The image decoding unit 211 receives a high resolution image from the image input unit 210 to decode the image, and decodes the decoded image to the low resolution panoramic image generating unit 214 or the high resolution digital PTZ image generating unit 219. to provide.

The image storage unit 212 receives the image stream from the image input unit 210 and stores it therein.

When a search for an image corresponding to a specific condition is requested, the image search / reproducing unit 213 reads out the image data corresponding to the specific condition from the image storage unit 212 and provides it to the image decoding unit 211. do. Here, the specific condition suggests a search condition based on event detection information (e.g., event detection such as roaming, virtual fence, backward movement, etc.) and object detection information (object information such as specific ^' color, size, etc.)'. Or a condition based on a date to be searched for.

<37> low-resolution panoramic image generation unit 214 using the low-resolution panoramic image generation table (2, ₁₅₎ with information of low-resolution panoramic image generation information to generate a low-resolution panoramic image. The low resolution panoramic image generating unit 214 provides the low resolution panoramic image to the object tracking unit 216 and the passive digital PTZ control unit 217, and to the image displaying unit 221 for displaying an image.

More specifically, the low resolution panoramic image generator 214 generates the low resolution panoramic image by using image warping and blending information stored in the low resolution panoramic image generation table 215. .

When the object tracking unit 216 receives the low resolution panoramic image from the low resolution panoramic image generating unit 214, the object tracking unit 216 detects the set event through object detection and tracking.

The object tracking unit 216 automatically digitally records the position and size information of the object for each frame. Provided to the PTZ control unit 218. Here, the method of detecting and tracking the object is omitted because it is outside the scope of the present invention. The passive digital image controller 217 converts the positional coordinate information designated by the user in the low resolution panoramic image into position coordinates of the high resolution panoramic image and provides the converted position information to the high resolution digital image generator 219.

<4ΐ> The automatic digital image control unit 218 converts the detected position coordinate information of the low resolution panoramic image detected by the object tracker 216 into the position coordinate of the high resolution panoramic image to generate the high resolution digital image image generator 219. To provide.

The high resolution digital image image generation unit 219 generates a high resolution digital image image corresponding to the positional coordinates of the received high resolution panoramic image and provides it to the image display unit 221. At this time, the high resolution digital image image generation unit In operation 219, a high resolution panoramic image is generated using only a part of the ultra-high resolution panoramic image generation table 220 without generating a full high resolution panoramic image.

The image display unit 221 displays an image received from the low resolution panoramic image generating unit 214 or the high resolution digital image generating unit 219.

<44>

3 is a flowchart illustrating an embodiment of a video surveillance method according to the present invention.

Referring to FIG. 3, the image monitoring apparatus 200 receives a high resolution image through the image input unit 210 from a plurality of high resolution cameras disposed in a radial form and a portion of the photographing area overlapped with each other (step S310). . The received high resolution image is typically encoded (compressed) through a codec such as Η.264.

The video surveillance apparatus 200 stores an image of a high resolution image received for each camera.

Save via 212 (step S320). At this time, the type of image data stored is a form of encoded image data received from a camera.

The video surveillance apparatus 200 decodes the image data received for each camera (step S330).

The video surveillance apparatus 200 generates a low resolution panoramic image through the low resolution panoramic image generating unit 214 from the decoded image data of each camera, and expresses the panoramic image through the image display unit 221. In this case, the low resolution panoramic image generator 214 generates a low resolution panoramic image from the high resolution camera images by using image warping and blending information stored in the low resolution panoramic image generation table 215. If a specific position or a specific moving object is designated in the low resolution panoramic image (step S350), the video surveillance apparatus 200 specifies the designated position or the designated movement through the manual digital PTZ controller 217 or the automatic digital PTZ controller 218. Obtain area information of the high resolution digital PTZ image corresponding to the position of the object (step S360).

In an embodiment of step S350, the user may designate a position corresponding to the ROI in the low resolution panoramic image being displayed.

In another embodiment of operation S350, the user may designate the object of interest among the moving objects being tracked through the object tracking unit 216.

In another embodiment of operation S350, the object location generating object or the object generating the detected event may be automatically designated as the location of interest or the object of interest through the object tracking unit 216.

The video surveillance apparatus 200 generates a high resolution digital PTZ image through the high resolution digital PTZ image generating unit 219 using the area information of the high resolution digital PTZ image acquired in step S360, and the image display unit 221. ) Through the screen. At this time, the high resolution digital PTZ image generating unit 219 uses the image warping and blending information stored in the ultra high resolution panoramic image generating table 220 to obtain a high resolution obtained from the image decoding unit 211. Create high resolution digital PTZ images from camera images.

<55>

4 is a flowchart illustrating an embodiment of a video surveillance method according to the present invention.

Referring to FIG. 4, the user requests the video surveillance apparatus 200 to play back a recorded video (step S410). The playback start position of the recorded video may be specified in various ways. For example, when a user directly designates a time point corresponding to a playback start position or when a user selects a specific event among previously generated events, the user may be designated as a start time of occurrence of the selected event.

The video surveillance apparatus 200 imports recorded video data for each camera recorded in the video storage unit 212 through the video search / playback unit 213 (step S420).

The video surveillance apparatus 200 is an image decoding unit for recording video data of each camera.

Decode via step 211 (step S430).

The video surveillance apparatus 200 generates a low resolution panoramic image through the low resolution panoramic image generator 214 from the decoded image data of each camera, and displays an image. The display on the screen via the unit 221 (step S440).

In order to view a high resolution image of the ROI or the moving object of interest, the user designates the corresponding position of the ROI or the moving object of interest in the low resolution panoramic image being displayed (step S450).

When the user designates a corresponding position of the ROI, the video surveillance apparatus 200 obtains region information of the corresponding high resolution digital PTZ image through the manual digital ΡΓΖ controller 217, and the user may move the ROI of interest. If it is specified, area information of the corresponding high resolution digital PTZ image is acquired through the automatic digital PTZ control unit 218 (step S460).

The video surveillance apparatus 200 performs a high resolution digital PTZ image generation unit 219 from the decoded image data of each camera acquired in step S430 and the region information of the high resolution digital PTZ image obtained in step S460. It generates a high-resolution digital PTZ image through, and is displayed on the screen through the image display unit 221.

<64>

<65> Figure 5 is a view for explaining a process of generating a low-resolution panoramic ^"video image monitoring apparatus 200 according to an embodiment of the present invention.

2 and 5, from the high resolution image of each camera acquired at the same time, the low resolution panoramic image generating unit 214 of the video surveillance apparatus 200 performs a low resolution panoramic image generation table 215. ) Produces a panoramic image of the same size (resolution) as a low resolution panoramic image generation table using image warping and blending information stored in the.

More specifically, the brightness value (R / G / B value in the case of a color image) of a specific pixel of the panorama image is obtained as a weighted sum of the pixel brightness values of the camera images. For example, suppose that the camera images II and 12 among the N camera images acquired from the N cameras have a pixel that corresponds to a specific pixel coordinate (x, y) of the panoramic image Ip (FIG. 5). Reference) . Then, the pixel brightness value Ip (x, y) at the pixel coordinates (x, y) of the panoramic image Ip is obtained by Equation 1 below.

Equation 1

I (x, y) = a (x _/ y) I ₁ (x _{1 /} y ₁ ) + (la (x _/ y)) I ₂ (x _{1 /} y ₁ )

<69>

In Equation 1, a (x, y) is a blending coefficient value at pixel coordinates (x, y) of the panorama image and has a value between 0 and 1. FIG. Pixel coordinates (，) and

Oc _{_2,} y ₂₎ is a panorama-image from the camera ₁₂ and to Daewoong the pixel coordinates of the image (x, y) Means pixel coordinate values. In other words, when the panorama image Ip is generated, the pixel coordinates (^^ and (x ₂ , y ₂ ) of the camera images 1 and 1 ₂ are warped to the pixel coordinates (x, y) of the panorama image Ip.

(warpi ng) means to be.

In the panorama image generation table, pixel coordinates and blending coefficient values of the camera image are stored for each pixel of the panorama image to be generated. Therefore, when N camera images acquired at the same time from N cameras are input, the entire panorama image or a portion thereof can be easily calculated using the panorama image generation table. The panorama image generation table can be obtained by a camera calibra- tion technique, and a detailed method is omitted since it is beyond the scope of the present patent.

The size of the low resolution panoramic image generation table is determined in consideration of the resolution of the panoramic image for output on a screen or for tracking an object. For example horizontally

When using five Ful l—HD (1920x1080) cameras, the resolution of the resulting panorama image is up to 8448x1080 (assuming that the overlap between neighboring camera images is 15%). However, the resolution of a typical display device (monitor) is about 1920x1080, which is much lower than this. Therefore, when displaying a panoramic image in a single line on the screen, the resolution of the display panoramic image is sufficient to be 1920x246. Therefore, in the above example, a low resolution panoramic image generation table having a size of 1920x246 may be provided.

<73>

FIG. 6 is a diagram for explaining a process of generating a high resolution digital PTZ image by the video surveillance apparatus 200 according to an embodiment of the present invention.

2 and 6, from the high resolution image of each camera acquired at the same time, the high resolution digital PTZ image generating unit 219 of the video surveillance apparatus 200 is a manual digital PTZ control unit 217 or the like. A high resolution digital PTZ image is generated by using the area information obtained from the automatic digital PTZ control unit 218 and image warping and blending information stored in the ultra-high resolution panoramic image generation table 220.

The structure of the ultra-high resolution panoramic image generation table 220 is the same as that of the low resolution panoramic image generation table 215, and the size of the table follows the maximum resolution of the panoramic image that can be generated through the original camera images. .

The high resolution digital PTZ image is generated by performing image warping and blending operations only within a corresponding region of the ultra high resolution panoramic image generation table. The size (resolution) of the digital PTZ image can be customized by the user.

<78>

While specific embodiments of the present invention have been described so far, various modifications may be made without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims below, but also by the equivalents of those claims.

As described above, the present invention has been described with reference to the limited embodiments and the drawings, but the present invention is not limited to the above embodiments, which can be obtained by those skilled in the art to which the present invention pertains. Various modifications and variations are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

<81

** Description of the symbols **

100: multiple high resolution cameras

200: video surveillance apparatus

210: image input unit

211: Video decoding section

212: Image storage unit

<88> 213: Image Search / Playback Section

214: low resolution panoramic image generating unit

215: low-resolution panoramic image generation table

216: object tracking unit

<217> 217: manual digital ΡΤΖ control unit

<218> 218: Automatic digital ΡΤΖ control unit

<94> 219: High resolution digital ΡΤΖ image.

220: Ultra-high resolution panoramic image generation table

221 ： Image Display Unit

Claims

[Range of request]

[Claim 11

In the video monitoring method is executed in a video surveillance device connected to a plurality of video recording devices arranged in a radial form and a portion of the shooting area overlap each other,

Receiving a high resolution image from each of the plurality of imaging apparatuses; Generating and displaying a low resolution panoramic image using the high resolution image based on a low resolution panoramic image generation table; And

And displaying a high resolution digital PTZ image using a high resolution panoramic image generation table when an event occurs in the process of displaying the low resolution panoramic image.

[Claim 2]

The method according to claim 1,

Generating and displaying a low resolution panoramic image using the high resolution image based on the low resolution panoramic image generation table

And generating and displaying a low resolution panoramic image from the high resolution camera images by using the image warping and blending information stored in the low resolution panoramic image generation table.

[Claim 3]

The method according to claim 1,

When an event occurs in the process of displaying the low resolution panoramic image, generating and displaying a high resolution digital PTZ image using a high resolution panoramic image generation table may include:

Acquiring region information of a high resolution digital PTZ image corresponding to the designated position or the position of the designated moving object when a specific position or a specific moving object is designated in the low resolution panoramic image;

Generating a high resolution digital PTZ image using area information of the high resolution digital PTZ image; And

And displaying the high resolution digital PTZ image.

[Claim 4]

The method according to claim 3, Displaying the high resolution digital PTZ image

Image warping stored in ultra high resolution panoramic image generation table

and generating and displaying a high-resolution digital PTZ image by using warping and blending information.

[Claim 5]

The method according to claim 1,

If a search for an image corresponding to a specific condition is requested, further comprising: reading, by the storage unit, an image corresponding to the specific condition;

The above specific conditions

And at least one of ^{: a} search condition based on event detection information, object detection information, or a search condition based on a date to be searched.

[Claim 6]

In the video monitoring device connected to a plurality of video recording devices arranged in a radial form and part of the shooting area overlap each other,

Image input unit for receiving a high resolution image from each of the plurality of image pickup devices

A low resolution panorama image generation unit generating a low resolution panorama image using the high resolution image based on the low resolution panorama image generation table and displaying the low resolution panorama image; And

Image monitoring includes a high resolution digital PTZ image generating unit for generating a high resolution digital PTZ image using a high resolution panoramic image generating table and displaying the same through the image display unit when an event occurs while the image display unit displays the low resolution panoramic image. Device .

[Claim 7]

The method according to claim 6,

The low resolution panoramic image generator

Image surveillance, characterized in that for generating a low resolution panoramic image from the high resolution camera images using the image warping and blending (bl ending) information stored in the low resolution panoramic image generation table to display through the image display unit Device .

[Claim 8] The method according to claim 6,

If a specific position or a specific moving object is designated by a user in the process of displaying the low resolution panoramic image, the image display unit acquires area information of a high resolution digital PTZ image corresponding to the designated position or the position of the designated moving object. Further includes an automatic digital PTZ control,

The high resolution digital PTZ image generating unit

And a high resolution digital PTZ image is generated using the region information of the high resolution digital PTZ image and displayed through the image display unit.

[Claim 9]

The method according to claim 8,

The high resolution digital PTZ image generating unit

An image surveillance apparatus comprising generating a high resolution digital PTZ image by using image warping and blending information stored in an ultra-high resolution panoramic image generation table and displaying the image through an image display unit.

[Claim 10]

The method according to claim 6,

The image storage unit may further include an image search unit for deriving an image corresponding to the specific condition when a search for an image corresponding to a specific condition is requested.

The above specific conditions

Video surveillance apparatus comprising at least one of a search condition based on the event detection information, object detection information or a search condition based on the date to be searched