KR100888935B1 - Method for cooperation between two cameras in intelligent video surveillance systems - Google Patents

Abstract

An interlocking method between two cameras in an intelligent digital video recording system for coupling a PTZ camera between a fixed camera and PTZ camera is provided to implement more efficient broad area monitoring operation by interworking a camera interval in the intelligent digital video recording system. A digital video recording system is the fixed camera A and fixed camera B, the respective still image is obtained in an intelligent digital video recording system(S101). A pixel coordinates corresponding to the arbitrary feature point is designated in the image of the fixed camera A(S102). In user is the image of the fixed camera B, the pixel coordinate corresponding to the feature point is designated(S103). The digital video recording system stores the pixel coordinates of the feature point designated from user in the form of the correspondence pair(S104). The digital video recording system determines whether the feature point more than 4 was designated from a user(S105).

Description

How to link two cameras in an intelligent video surveillance system {METHOD FOR COOPERATION BETWEEN TWO CAMERAS IN INTELLIGENT VIDEO SURVEILLANCE SYSTEMS}

The present invention relates to an interlocking method between two cameras in an intelligent video surveillance system supporting a plurality of cameras including a fixed camera and a PTZ camera for wide area surveillance.

In general, a closed-circuit television (CCTV) system is a passive surveillance system that relies solely on human interpretation of images from CCTV cameras. Intelligent video surveillance systems are being introduced to detect and track moving objects by analyzing them in real time, and to detect meaningful events based on them.

Most of these intelligent video surveillance systems have the ability to handle multiple cameras at the same time, but the processing of each camera image is completely independent and the interworking between cameras is not considered.

However, in the case of wide-area surveillance that monitors a large area with multiple cameras, more effective surveillance can be achieved through interworking between the cameras. Examples of interlocking between cameras are when the object being tracked by the fixed camera A enters the surveillance area of the neighboring fixed camera B through the surveillance area of the fixed camera A, or when the fixed camera B tracks the object or detects the fixed camera. Another example is receiving a PTZ object from a neighboring PTZ camera and automatically performing PTZ tracking.

As a prior art related to interworking between cameras, Korean Patent No. 10-0599587 discloses a camera calibrating method for interworking between a fixed camera and a PTZ camera in the “Camera calibrating method for automatically expanding the PTZ camera view into a fixed camera view”. However, the method using the interpolation and lookup table used in the prior art has to specify a large number of representative points in order to obtain the pan / tilt angle of the PTZ camera corresponding to every pixel of the fixed camera image. In addition, there is a problem that the prior art is limited to the interworking of the fixed camera and the PTZ camera.

Meanwhile, the "Wide-Area Site-Based Video Surveillance System" of the international patent registration number WO 2008/054489 discloses a method of interlocking a plurality of cameras for wide area surveillance, but the prior art indicates the entire surveillance area. There is a problem in that accurate maps or photographs are required.

An embodiment of the present invention provides an interlocking method between a fixed camera and a fixed camera, between a fixed camera and a PTZ camera, and between a PTZ camera and a PTZ camera in an intelligent video surveillance system supporting a plurality of cameras including a fixed camera and a PTZ camera. do.

As a technical means for achieving the above technical problem, the first aspect of the present invention is a method for interlocking between fixed cameras in an intelligent video surveillance system, (a) for the predetermined number or more of the feature points present on the ground, the feature points Performing camera correction between the first camera and the second camera from corresponding pairs of pixel coordinates in the image of the first fixed camera and pixel coordinates in the image of the second fixed camera corresponding to (b) And providing a corresponding relationship between an object being tracked in the first fixed camera image and an object being tracked in the second fixed camera image by using the camera correction result.

As a technical means for achieving the above technical problem, the second aspect of the present invention is a method for interlocking between a fixed camera and a PTZ camera in an intelligent video surveillance system, (a) for a predetermined number or more of the feature points present on the ground, (B) performing camera calibration between the fixed camera and the PTZ camera from corresponding pairs of pixel coordinates and pan / tilt angles of the PTZ camera in the fixed camera image corresponding to the feature points; and (b) the camera. A method of interlocking between a fixed camera and a PTZ camera, the method comprising controlling the PTZ camera to look at an object being tracked in the fixed camera image by using a correction result.

As a technical means for achieving the above technical problem, the third aspect of the present invention is a method for interlocking PTZ cameras in an intelligent video surveillance system, (a) for the predetermined number or more of the feature points present on the ground, the feature points Performing camera calibration between the first PTZ camera and the second PTZ camera from corresponding pairs of pan / tilt angle coordinates of the first PTZ camera and pan / tilt angle coordinates of the second PTZ camera corresponding to and b) controlling the second PTZ camera to look at the object being tracked in the first PTZ camera image using the camera calibration result.

According to the above-described problem solving means of the present invention, it is possible to implement an intelligent video surveillance system that supports the inter-camera for more efficient wide area monitoring.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, which means that it may further include other components, except to exclude other components unless otherwise stated.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a general configuration diagram of an intelligent video surveillance system using multiple cameras.

As shown in FIG. 1, the intelligent video surveillance system 100 includes a plurality of fixed cameras 101 and 102, a plurality of PTZ cameras 103 and 104, image processing devices 105, 106, 107, and 108, wired and wireless devices. Communication network 109 and central control unit 110.

The fixed cameras 101 and 102 capture an image input from a lens through an image pickup device such as a CCD or a CMOS and transmit the image to the image processing apparatus 105 or 106.

PTZ (Pan, Tilt, Zoom) cameras 103 and 104 capture an image input from a lens through an imaging device such as a CCD or a CMOS and transmit the image to the image processing apparatuses 107 and 108. In addition, the PTZ cameras 103 and 104 are video cameras equipped with an electric zoom lens and are attached to a pan / tilt driving device, and are controlled by the control signals from the image processing devices 107 and 108. Rotation and zoom in / zoom out of the lens are possible.

The wired / wireless communication network 109 may be a wired network, a mobile radio communication network, a satellite, such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN). It can be implemented in all kinds of wireless networks such as communication networks, Bluetooth, Wireless Broadband Internet (Wibro), High Speed Downlink Packet Access (HSDPA), and the like.

The image processing apparatuses 105, 106, 107, and 108 receive images from the fixed cameras 101 and 102 or the PTZ cameras 103 and 104, analyze them in real time, detect / track / classify moving objects, and the like. Detects the event specified by the user, and when the PTZ cameras 103 and 104 are connected, PTZ / 103 tracking of the moving object is performed by automatically controlling pan / tilt / zoom of the PTZ cameras 103 and 104. In addition, the image processing apparatus 105, 106, 107, 108 is a wired / wireless communication network 109 for the image received from the fixed camera (101, 102) or PTZ cameras (103, 104), moving object detection / tracking and event detection results, etc. It is transmitted to the central control unit 110 through), and receives and performs a variety of settings and control commands input by the user through the central control unit (110).

The central control unit 110 is connected to the plurality of image processing units 105, 106, 107, 108 through the wired / wireless communication network 109, and manages the image processing units 105, 106, 107, 108. do. The central control unit 110 receives an image, a moving object detection / tracking, and an event detection result from the image processing apparatuses 105, 106, 107, and 108 and displays the result on a screen, and stores the result in a local storage device if necessary. The stored data is used for future data retrieval and playback. In addition, the central control unit 110 receives various setting and control commands from the user and transmits them to the image processing apparatuses 105, 106, 107, and 108, and the cameras through the image processing apparatuses 105, 106, 107, and 108. Involved in interlocking Reference numerals denoted by the person 120, the pixel coordinates 121, the ground surface 130, and the feature point 131 which are being tracked will be described later.

Here, the interworking between the cameras can be divided into the following three.

The first is the linkage between fixed camera A and fixed camera B. When the moving object being tracked in fixed camera A moves past fixed camera A to fixed camera B, the fixed camera B recognizes that it is the same object and then tracks it.

Second, the PTZ camera automatically performs PTZ tracking on the moving object detected or being tracked by the fixed camera.

Third, the PTZ camera B automatically performs PTZ tracking on the moving object detected or being tracked by the PTZ camera A, which stays at an arbitrary position, by linking the PTZ camera and the PTZ camera.

Hereinafter, a method of interworking between two cameras in an intelligent video surveillance system according to an embodiment of the present invention will be described with reference to FIGS. 2 to 7.

2 and 3 are flow charts for explaining a method of interlocking a fixed camera and a fixed camera according to an embodiment of the present invention, respectively.

FIG. 2 is a flowchart illustrating a camera correction method for interworking between the fixed camera A 101 and the fixed camera B 102. The camera correction according to FIG. 2 is performed by the intelligent video surveillance system 100 to actually monitor video. It is done before execution, and it is the part that needs manual operation by user.

In step S101, the video surveillance system 100 acquires still images from the fixed camera A 101 and the fixed camera B 102 and displays them on the screen for the user to confirm. At this time, as in the example of FIG. 1, it is assumed that there is an overlapping portion between the view of the fixed camera A 101 and the view of the fixed camera B 102.

에 대응하는 픽셀 좌표를 지정한다. 특징점

에 대응하는 고정식 카메라 A(101) 영상에서의 픽셀 좌표를

라고 하자. 도 1에서 도면 부호 130은 지면(Ground)을 나타내며, 도면 부호 131는 지면 위에 존재하는 특징점(Feature Point)을 나타낸다. 이때, 지면(130)은 거의 평평하다고 가정하며, 특징점(131)으로는 지면(130) 위에 존재하는, 영상에서 구분이 가능한 점(Point)이면 무엇이든 상관없다. 특징점의 예로, 건물의 모퉁이가 지면과 만나는 곳 또는 가로등이나 전신주가 지면과 만나는 곳 등을 들 수 있다. 사용자가 고정식 카메라 A(101)의 영상에서 특징점에 해당하는 픽셀 좌표를 지정할 때, 고정식 카메라 B(102)의 영상에서도 공통으로 보이는 특징점을 사용해야 한다.In step S102, the user selects any feature point present on the ground in the image of the fixed camera A 101.

Specifies the pixel coordinates corresponding to. Feature Point

The pixel coordinates in the fixed camera A 101 image corresponding to

Let's say In FIG. 1, reference numeral 130 denotes a ground, and reference numeral 131 denotes a feature point existing on the ground. In this case, it is assumed that the ground 130 is almost flat, and the feature point 131 may be any point that can be distinguished from an image existing on the ground 130. An example of a feature point is where a corner of a building meets the ground or where a street lamp or telephone pole meets the ground. When the user designates pixel coordinates corresponding to the feature points in the image of the fixed camera A 101, the feature points commonly seen in the image of the fixed camera B 102 should be used.

에 대응하는 픽셀 좌표를 지정한다. 이때, 지정된 픽셀 좌표를

라고 하자.In step S103, the user selects a feature point in the image of the fixed camera B102.

Specifies the pixel coordinates corresponding to. Where the specified pixel coordinates

Let's say

의 픽셀 좌표들을 대응쌍(Correspondence Pair)의 형태인

로 저장한다.In step S104, the video surveillance system 100 receives the feature points designated by the user in steps S102 and S103.

Pixel coordinates in the form of a Correspondence Pair

Save as.

In step S105, the video surveillance system 100 determines whether four or more feature points have been designated by the user. If not, the flow returns to step S102. That is, four or more feature points are specified by repeating from step S102 to step S104.

(i = 1,2,3,4,...)을 이용하여 카메라 보정(Camera Calibration)을 수행한 다음, 단계(S107)로 진행하여, 그 결과로서 호모그래피 행렬(Homography Matrix)

을 얻는다.In step S106, the video surveillance system 100 corresponds to a pair of pixel coordinates for the stored four or more feature points.

Perform Camera Calibration using (i = 1,2,3,4, ...), then proceed to step S107, as a result of the Homography Matrix

Get

은 지면(130) 상에 있는 임의의 특징점에 대하여 고 정식 카메라 A(101) 영상과 고정식 카메라 B(102) 영상에서의 픽셀 좌표의 변환 관계를 나타내는 3 x 3의 행렬이다. 즉, 지면(130) 상에 있는 임의의 특징점에 대하여 고정식 카메라 A(101) 영상에서의 픽셀 좌표를

라고 하고, 고정식 카메라 B(102) 영상에서의 픽셀 좌표를

라고 하면, 하기 수학식 1과 같은 관계가 성립한다.Homography matrix

Is a matrix of 3 x 3 representing the transformation relationship between pixel coordinates in the fixed camera A 101 image and the fixed camera B 102 image for any feature point on the ground 130. That is, the pixel coordinates of the fixed camera A 101 image for any feature point on the ground 130 are adjusted.

The pixel coordinates in the fixed camera B 102 image

In this case, the same relationship as in Equation 1 below is established.

의 (i,j) 번째 요소를

라고 표시하면, 상기 수학식 1로부터

는 하기 수학식 2와 같이 주어진다.In Equation 1, it is an arbitrary real value other than zero. procession

Element of (i, j)

If expressed as, from Equation 1

Is given by Equation 2 below.

(i = 1,2,3,4,...) 이 주어지면, 행렬

은 하기 수학식 3의 함수

의 값을 최소화하는

를 계산함에 의해 구해진다.Pair of pixel coordinates of four or more feature points

Given (i = 1,2,3,4, ...), the matrix

Is a function of Equation 3

To minimize the value of

Obtained by calculating

FIG. 3 shows a fixed camera A in order to recognize and subsequently track the object being tracked by the fixed camera A 101 when the intelligent video surveillance system 100 actually performs the video surveillance. FIG. 101 is a flowchart illustrating a method of obtaining a correspondence relationship between objects being tracked in the image of the fixed camera B 102.

In operation S201, the video surveillance system 100 performs object tracking on the fixed camera A 101 image and the fixed camera B 102 image, respectively.

을 획득한다. 예를 들어, 도 1에서 고정식 카메라 A(101)를 통해 추적 중인 사람(120)이 객체 A에 해당한다면, 고정식 카메라 A 영상에서 121에 해당하는 곳에 대한 픽셀 좌표가

가 된다.In operation S202, the video surveillance system 100 may be configured to, for any object A being tracked in the fixed camera A 101 image, pixel coordinates where the object A contacts the ground.

Acquire. For example, in FIG. 1, if the person 120 being tracked through the fixed camera A 101 corresponds to the object A, the pixel coordinates of the place corresponding to 121 in the fixed camera A image are shown.

Becomes

및 수학식 2를 이용하여

와 대응하는 고정식 카메라 B(102) 영상에서의 픽셀 좌표

를 구한다.In step S203, the video surveillance system 100 obtains a homography matrix obtained through the camera calibration.

And using Equation 2

Pixel coordinates in the fixed camera B 102 image corresponding to

Obtain

와 일정 거리 이내에 있으면서

와 가장 가까운 객체를 찾는다. 상기 찾은 객체를 객체 B라고 하면, 영상 감시 시스템(100)은 고정식 카메라 B(102)에서 추적 중인 객체 B를 고정식 카메라 A(101)에서 추적 중인 객체 A와 동일한 객체로 인지하고 객체 A와 동일한 ID를 객체 B에 부여한다.In step S204, the video surveillance system 100 is the pixel coordinate of the position where the object and the ground touches among the objects being tracked in the fixed camera B (102) image

Within a certain distance of

Find the object nearest to. If the found object is called object B, the video surveillance system 100 recognizes the object B being tracked by the fixed camera B 102 as the same object as the object A being tracked by the fixed camera A 101 and has the same ID as the object A. To object B.

4 and 5 are flowcharts for explaining a method of interlocking a fixed camera and a PTZ camera according to an embodiment of the present invention, respectively.

4 is a flowchart illustrating a camera calibration method for interworking between the fixed camera 102 and the PTZ camera 103. The camera calibration according to FIG. 4 is performed before the intelligent video surveillance system 100 actually performs video surveillance. This is a part that requires manual operation by the user.

When the camera calibration step is entered, the video surveillance system 100 obtains a still image from the fixed camera 102 and displays it on the screen for the user to check.

에 대응하는 픽셀 좌표를 지정한다. 특징점

에 대응하는 고정식 카메라(102) 영상에서의 픽셀 좌표를

라고 하자.In step S302, the user is in the image of the fixed camera 102, any feature point present on the ground

Specifies the pixel coordinates corresponding to. Feature Point

The pixel coordinates in the fixed camera 102 image corresponding to

Let's say

에 일치시킨다. In step S303, the user manually controls the PTZ camera 103 to mark the center of the PTZ camera image.

To match.

라고 하자.In operation S304, the video surveillance system 100 reads the current pan / tilt angle coordinates from the PTZ camera 103. At this time, read the pan / tilt coordinates

Let's say

로 저장한다.In step S305, the video surveillance system 100 converts the pixel coordinates and pan / tilt angular coordinates obtained in steps S302 and S304 into a corresponding pair.

Save as.

In step S306, the video surveillance system 100 confirms that four or more feature points have been designated, and if four or more feature points have been designated, the process proceeds to step S307, while if four or more feature points are not specified, the step ( Return to S302). That is, four or more feature points are specified while repeating from step S302 to step S305.

(i = 1,2,3,4,...)을 이용하여 카메라 보정을 수행한 다음, 단계(S308)로 진행하여, 그 결과로써 호모그래피 행렬

를 얻는다.In step S307, the video surveillance system 100 corresponds to a pair of pixel coordinates and pan / tilt angle coordinates for four or more stored feature points.

Perform camera calibration using (i = 1,2,3,4, ...) and then proceed to step S308 whereby the homography matrix

Get

는 지면 상에 있는 임의의 특징점에 대하여 고정식 카메라 영상에서의 대응되는 픽셀 좌표와, PTZ 카메라가 동일한 특징점을 바라볼 때의 방향을 나타내는 3D 방향 벡터 사이의 변환 관계를 나타내는 3 x 3의 행렬이다. 즉, 지면 상에 있는 임의의 특징점에 대하여 고정식 카메라 영상에서의 픽셀 좌표를

라고 하고, 동일한 특징점을 바라볼 때의 PTZ 카메라의 팬/틸트 각 좌표를

라고 하면, 하기 수학식 4와 같은 관계가 성립한다.Homography matrix

Is a matrix of 3 x 3 representing the transformation relationship between the corresponding pixel coordinates in the stationary camera image for any feature point on the ground and the 3D direction vector representing the direction when the PTZ camera looks at the same feature point. That is, the pixel coordinates in the fixed camera image for any feature point on the ground

And pan / tilt coordinates of the PTZ camera when looking at the same feature point.

In this case, the relationship as shown in Equation 4 below is established.

는 하기 수학식 5와 같이 주어진다.3D direction vector of the PTZ camera in Equation 4

Is given by Equation 5 below.

(i = 1,2,3,4,...)이 주어지면, 행렬

는 하기 수학식 6의 함수

의 값을 최소화하는

를 계산함에 의해 구해진다.Correspondence pairs for four or more feature points

Given (i = 1,2,3,4, ...), the matrix

Is a function of Equation 6

To minimize the value of

Obtained by calculating

5 shows the pan / PTZ of the PTZ camera 103 so that when the intelligent video surveillance system 100 actually performs video surveillance, the PTZ camera 103 is directed toward the specific object being tracked by the fixed camera 102. It is a flowchart explaining the method of obtaining and moving a tilt angle value.

In operation S401, the video surveillance system 100 performs tracking of a specific object in the fixed camera image. Call the object you are tracking object A.

를 구한다.In operation S402, the video surveillance system 100 may determine pixel coordinates where the object A contacts the ground in the fixed camera image.

Obtain

및 상기 수학식 4를 이용하여 3D 벡터

를 구한다. 이 때

로 놓는다.In step S403, the video surveillance system 100 obtains a homography matrix obtained through the camera calibration.

And a 3D vector using Equation 4 above.

Obtain At this time

Place it.

로부터 PTZ 카메라(103)가 객체 A를 향하게 만들기 위한 팬/틸트 각

를 하기 수학식 7을 통하여 구한다.In operation S404, the video surveillance system 100

Pan / tilt angle to direct PTZ camera 103 towards object A

Is obtained through Equation 7 below.

In operation S405, the video surveillance system 100 moves the PTZ camera to the calculated pan / tilt angle value.

6 and 7 are flowcharts illustrating a method of interlocking a PTZ camera and a PTZ camera according to an embodiment of the present invention.

6 is a flowchart illustrating a camera calibration method for interworking between the PTZ camera A 103 and the PTZ camera B 104. The camera calibration according to FIG. 6 is performed before the intelligent video surveillance system 100 actually performs video surveillance. This is the part that needs manual operation by user.

를 고려한다.When entering the camera calibration step, in step S501 the user has a suitable feature point present on the ground within the overlapping surveillance area of PTZ camera A and PTZ camera B.

Consider.

에 일치시킨다.In step S502, the user manually controls the PTZ camera A 103 to characterize the center of the PTZ camera image.

To match.

에 일치시킨다.In step S503, the user manually controls the PTZ camera B 104 to characterize the center of the PTZ camera image.

To match.

와 PTZ 카메라 B(104)의 현재 팬/틸트 각 좌표

를 읽어 와 대응쌍의 형태인

로 저장한다.In operation S504, the video surveillance system 100 may coordinate the current pan / tilt angle of the PTZ camera A 103.

Current pan / tilt coordinates of PTZ camera B (104)

Read and match the form

Save as.

In step S505, the video surveillance system 100 confirms that four or more feature points have been designated, and if four or more feature points are specified, proceeds to step S506, and if four or more feature points are not specified, step S501. Return to. That is, four or more feature points are specified while repeating from step S501 to step S504.

(i = 1,2,3,4,...)을 이용하여 카메라 보정을 수행한 다음, 단계(S507)로 진행하여, 그 결과로써 호모그래피 행렬

을 얻는다.In step S506, the video surveillance system 100 corresponds to a corresponding pair of pan / tilt angular coordinates for the stored four or more feature points.

Perform camera calibration using (i = 1,2,3,4, ...) and then proceed to step S507 whereby the homography matrix

Get

은 지면 상에 있는 임의의 특징점

에 대하여 PTZ 카메라 A(103)가

를 바라볼 때의 방향을 나타내는 3D 벡터

와 PTZ 카메라 B(104)가

를 바라볼 때의 방향을 나타내는 3D 벡터

사이의 변환 관계를 나타내는 3 x 3의 행렬이다. 즉, 벡터

와 벡터

사이에는 하기 수학식 8과 같은 관계가 성립한다.Homography matrix

Is any feature point on the ground

Against PTZ Camera A (103)

3D vector representing the direction when looking at

And PTZ camera B (104)

3D vector representing the direction when looking at

A 3 x 3 matrix representing the transformation relationship between I.e. vector

Vector with

The following relationship holds for the following equation (8).

와

는 하기 수학식 9와 수학식 10과 같이 주어진다.In Equation 8

Wow

Is given by Equations 9 and 10 below.

(i = 1,2,3,4,...)이 주어지면, 행렬

은 하기 수학식 11의 함수

의 값을 최소화하는

를 계산함에 의해 구해진다.Correspondence pairs for four or more feature points

Given (i = 1,2,3,4, ...), the matrix

Is a function of Equation 11

To minimize the value of

Obtained by calculating

7 shows that when the intelligent video surveillance system 100 actually performs video surveillance, the PTZ camera B 104 is directed toward the specific object being tracked in the image of the PTZ camera A 103 which stays at a specific position. It is a flowchart illustrating a method of obtaining and moving a pan / tilt angle value of the PTZ camera B 104 so as to move.

라고 하고, 추적 중인 특정 객체를 객체 A라고 하자.In operation S601, the video surveillance system 100 performs tracking of a specific object in the image of the PTZ camera A 103 remaining at a specific location. Coordinates of the current pan / tilt angle of the PTZ camera A (103)

Let's call the specific object being tracked object A.

를 구한다In operation S602, the video surveillance system 100 may determine pixel coordinates where the object A contacts the ground in the image of the PTZ camera A 103.

Find

로부터 객체 A가 지면과 닿는 위치에 대한 3D 방향 벡터

를 구한다. 벡터

는 하기 수학식 12와 같이 주어진다.In step S603, the video surveillance system 100

3D direction vector of where object A touches the ground from

Obtain vector

Is given by Equation 12 below.

와 회전 행렬

은 각각 하기 수학식 13과 수학식 14와 같이 주어진다.Vector in Equation 12

And rotation matrix

Are given by Equations 13 and 14, respectively.

는 PTZ 카메라 A(103)의 렌즈 초점 거리(Focal Length) 값이다.In Equation 13

Is a lens focal length value of the PTZ camera A 103.

및 상기 수학식 8을 이용하여 3D 벡터

를 구한다. 이 때

로 놓는다.In step S604, the video surveillance system 100 obtains a homography matrix obtained through the camera calibration.

And a 3D vector using Equation 8

Obtain At this time

Place it.

로부터 PTZ 카메라 B(104)가 객체 A를 향하게 만들기 위한 팬/틸트 각

를 하기 수학식 15를 통하여 구한다.In step S605 the video surveillance system 100

Pan / tilt angle to direct PTZ camera B 104 from the camera toward object A

Is obtained through Equation 15 below.

In step S606, the video surveillance system 100 moves the PTZ camera B 104 to the value of the calculated pan / tilt angle.

The system and method according to an embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

1 is a general configuration diagram of an intelligent video surveillance system using multiple cameras.

2 and 3 are flow charts for explaining a method of interlocking a fixed camera and a fixed camera according to an embodiment of the present invention, respectively.

4 and 5 are flowcharts for explaining a method of interlocking a fixed camera and a PTZ camera according to an embodiment of the present invention, respectively.

6 and 7 are flowcharts illustrating a method of interlocking a PTZ camera and a PTZ camera according to an embodiment of the present invention.

Claims (9)

In the interlocking method between the fixed camera in the intelligent video surveillance system, (a) for a predetermined number or more of the feature points present on the ground, from the corresponding pairs of pixel coordinates in the image of the first fixed camera and pixel coordinates in the image of the second fixed camera corresponding to the feature points; Performing camera calibration between a camera and the second camera; and (b) obtaining a correspondence relationship between the object being tracked in the first fixed camera image and the object being tracked in the second fixed camera image using the camera calibration result; Linkage method between the fixed camera comprising a. The method of claim 1, In step (a), (a-1) designating pixel coordinates corresponding to the feature points in the still image obtained from the first fixed camera with respect to a predetermined number or more of the feature points existing on the ground; (a-2) designating pixel coordinates corresponding to the feature points in the still image obtained from the second fixed camera with respect to the feature points selected in step (a-1); and (a-3) obtaining a homography matrix between the first fixed camera and the second fixed camera from corresponding pairs of pixel coordinates of the feature points specified in steps (a-1) and (a-2) step  Interlocking method between the fixed camera that further comprises. The method of claim 1, In step (b), (b-1) performing object tracking on the image of the first fixed camera to obtain pixel coordinates of the ground on which the object being tracked is located; (b-2) Pixel coordinates corresponding to the pixel coordinates of the tracking object obtained in the step (b-1) using the homography matrix between the first fixed camera and the second fixed camera in the second fixed camera image Obtaining and (b-3) By performing object tracking on the image of the second fixed camera, the pixel coordinates of the positions of contacting the ground among the one or more objects being tracked are within a certain distance from the pixel coordinates obtained in step (b-2). Steps to Select Existing Objects Interlocking method between the fixed camera that further comprises. In the interworking method between a fixed camera and a PTZ camera in an intelligent video surveillance system, (a) the fixed camera and the PTZ from a corresponding pair of pixel coordinates and a pan / tilt angle of the PTZ camera in the fixed camera image corresponding to the feature points for a predetermined number or more of the feature points existing on the ground; Performing camera calibration between cameras, and (b) controlling the PTZ camera to look at the object being tracked in the fixed camera image by using the camera correction result; Linkage method between the fixed camera and a PTZ camera comprising a. The method of claim 4, wherein In step (a), (a-1) specifying pixel coordinates corresponding to the feature points in a still image obtained from the fixed camera, for a predetermined number or more of feature points existing on the ground; (a-2) acquiring pan / tilt angle coordinates of the PTZ camera whose center of the image of the PTZ camera corresponds to the feature point with respect to the feature point selected in step (a-1); and (a-3) Homo between the fixed camera and the PTZ camera from corresponding pairs of pixel coordinates and pan / tilt angle coordinates for the feature points specified in steps (a-1) and (a-2) above Steps to find the graphics matrix How to further interwork between the fixed camera and the PTZ camera. The method of claim 4, wherein In step (b), (b-1) performing object tracking on the image of the fixed camera to obtain pixel coordinates of the ground on which the object being tracked is located; (b-2) By using a homography matrix between the fixed camera and the PTZ camera from corresponding pairs of the fixed camera pixel coordinates and the pan / tilt angle coordinates of the PTZ camera, the tracking object obtained in the step (b-1) Obtaining a 3D direction vector of the PTZ camera corresponding to the pixel coordinates, (b-3) obtaining pan / tilt angle coordinates of the PTZ camera from the 3D direction vector of the PTZ camera obtained in the step (b-2); and (b-4) moving the PTZ camera to the pan / tilt angle coordinates obtained in the step (b-3); How to further interwork between the fixed camera and the PTZ camera. In the method of interworking between PTZ cameras in an intelligent video surveillance system, (a) For a predetermined number or more of the feature points present on the ground, the first from a corresponding pair of pan / tilt angle coordinates of the first PTZ camera and pan / tilt angle coordinates of the second PTZ camera corresponding to the feature points. Performing camera calibration between the PTZ camera and the second PTZ camera; and (b) controlling the second PTZ camera to look at the object being tracked in the first PTZ camera image using the camera calibration result; Linkage method between the PTZ camera comprising a. The method of claim 7, wherein In step (a), (a-1) acquiring pan / tilt angular coordinates of the first PTZ camera whose center of the image of the first PTZ camera corresponds to the feature points with respect to a predetermined number of feature points existing on the ground; (a-2) acquiring pan / tilt angle coordinates of the second PTZ camera whose center of the image of the second PTZ camera corresponds to the feature point with respect to the feature point selected in step (a-1); and (a-3) between the first PTZ camera and the second PTZ camera from corresponding pairs of pan / tilt angular coordinates for the feature points specified in steps (a-1) and (a-2); Steps to find a homography matrix How to link between the PTZ camera further comprising. The method of claim 7, wherein In step (b), (b-1) performing object tracking on the image of the first PTZ camera staying at an arbitrary place to obtain pixel coordinates of the tracking object corresponding to a position where the object being tracked touches the ground; (b-2) obtaining a 3D direction vector on the first PTZ camera coordinate system corresponding to the pixel coordinates of the tracking object obtained in the step (b-1); (b-3) using a homography matrix between the first PTZ camera and the second PTZ camera from corresponding pairs of pan / tilt angle coordinates of the first PTZ camera and pan / tilt angle coordinates of the second PTZ camera; obtaining a 3D direction vector of the second PTZ camera corresponding to the 3D direction vector obtained in step (b-2), (b-4) obtaining pan / tilt angle coordinates of the second PTZ camera from the 3D direction vector of the second PTZ camera obtained in the step (b-3); and (b-5) moving the second PTZ camera to the pan / tilt angle coordinates obtained in the step (b-4); How to link between the PTZ camera further comprising.

Images