WO2021253961A1

WO2021253961A1 - Intelligent visual perception system

Info

Publication number: WO2021253961A1
Application number: PCT/CN2021/087921
Authority: WO
Inventors: 李丰; 万成凯
Original assignee: 北京世纪瑞尔技术股份有限公司
Priority date: 2020-06-15
Filing date: 2021-04-16
Publication date: 2021-12-23
Also published as: CN111770266A; CN111770266B

Abstract

The present invention provides an intelligent visual perception system, comprising a variable focal length camera and a false target feedback feature information database in a designated monitoring area. The camera acquires video images of the monitoring area at a first resolution, and primary target detection in the area is performed by using a primary target detection algorithm, the primary target detection comprising: determining, on the basis of the false target feedback feature information database, whether a suspected target is contained in the designated monitoring area; when it is found that a suspected target is contained in the monitoring area, the camera is adjusted to acquire video images of the suspected target at a second resolution, and secondary target detection is performed using a secondary target detection algorithm to determine whether the suspected target is a true target or a false target; and if the suspected target is a false target, the false target feedback feature information database is updated according to information of the false target, or if the suspected target is a true target, the true target is tracked and monitored. The present intelligent visual perception system improves the precision and accuracy in capturing environmental anomalies.

Description

An intelligent visual perception system

Technical field

The invention relates to the field of security and protection, in particular to an intelligent visual perception system.

Background technique

Common movable cameras include pan-tilt cameras, PTZ cameras, etc., which usually have three control parameters of horizontal rotation, vertical rotation and focal length change. By adjusting these parameters, the motion camera can not only change the focal length to obtain different resolution information of the target or area in the shooting scene, but also change the angle to obtain different field of view direction information of the target or area in the shooting scene. However, these cameras, when shooting scenes in different areas, use the pan-tilt to control the direction of the camera's field of view, and the field of view direction is preset. This method only realizes an automatic monitoring.

A multi-camera is a commonly used imaging device. The body of the multi-camera is equipped with multiple lenses, and the multiple lenses are distributed along the circumferential direction of the body. The multiple lenses of the current multi-eye camera are usually fixed on the base to perform shooting in various angles and directions. The images taken by multiple lenses can be combined into a panoramic image of a multi-lens camera. However, at present, the main function of these multi-eye cameras is to achieve panoramic azimuth video shooting, and the discoverable distance of the target and the recognizable distance of the target are both small. If the focal lengths of the lenses of these multi-cameras are all lengthened, these multi-cameras are relatively large in size and not suitable for actual scene applications.

Most of the existing video surveillance systems are passive surveillance. Such a system is basically "monitoring but not controlling", and it cannot respond quickly when problems occur; it cannot collect video data selectively and intelligently. The system consumes a lot of resources, but its efficiency is very low.

When monitoring a large area, although the wide-angle high-definition panoramic camera can see the entire width of the scene, the distance that the camera can see is very limited. When there is a target in the video that needs to be monitored, the details of the target can only be found and found from the direction of the panoramic field of view. In this case, generally only rough information can be obtained. The system cannot automatically capture, track, amplify and collect these targets automatically, and cannot obtain detailed information about the targets.

Although there are a few intelligent video systems that can intelligently identify and track targets, they can only be performed on a single area, and the scope, efficiency, and accuracy of the monitoring area are not ideal. At the same time, due to the lack of accurate judgment of the suspected target, additional actions of the system are generated. At the same time, there is no secondary verification of the true and false targets, resulting in a lot of false alarm information. The processing time has caused this type of security system to fail to play its true role, resulting in irreparable losses.

Summary of the invention

In order to overcome the shortcomings of the prior art, the present invention provides an intelligent visual perception system that can find targets in real time, track targets, recognize targets, and alert in time, improve the accuracy and accuracy of the system in capturing environmental abnormalities, and make up for the prior art The shortcomings.

The present invention provides an intelligent visual perception system, wherein the system includes a variable focal length camera and a false target feedback feature information database in a designated monitoring area;

The camera collects video images of the monitored area at the first resolution, and uses a target detection algorithm for the collected video images to perform a target detection in the area, including: judging whether the designated monitoring area contains suspected targets based on the false target feedback feature information database ；

When it is found that the monitoring area contains a suspected target, adjust the camera to collect a video image of the suspected target at the second resolution, and based on the collected video image, use a secondary target detection algorithm to perform secondary target detection to determine the suspected target as True target or false target;

When the suspected target is a false target, update the false target feedback feature information database according to the information of the false target;

When the suspected target is a true target, follow up and monitor the true target;

The second resolution is greater than the first resolution.

The invention detects the valuable target in the area twice, can take into account the contradiction between the larger scene of the monitoring area and the target details, and can detect the suspicious target in the larger monitoring area through one target detection, with large resolution The detection results on the rate images often have high accuracy, and reliable detection results can be obtained through secondary target detection.

Preferably, the target feedback feature information database includes location information of the fake target and feature description information of the fake target;

The video image collection of the monitoring area by the camera at the first resolution includes: the video image collection of the monitoring area by the camera with a large field of view and a small resolution;

A target detection includes: based on the false target feedback feature information database, judging whether the image features in the monitoring area meet the specified conditions with the false target feedback feature description and location information matching degree of the area. If it is met, it is not judged as a suspected target; The target detection algorithm includes a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background;

When it is found that the monitoring area contains one or more suspected targets, adjust the camera's field of view direction and angle of view, and perform video image acquisition on the suspected target at the second resolution, including: successively performing the suspected target with a small field of view and a large resolution Video image collection;

The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background;

Updating the false target feedback feature information database according to the information of the false target includes: mapping the false target determined under the large resolution image to the small resolution image, obtaining the position information of the false target, extracting the description information of the false target feature, and updating the false target Feedback feature information database;

When the suspected target is a real target, track and monitor the real target, and output large-resolution video data, output alarm information and/or target characteristic information;

Large field of view and small resolution means that the video image covers the area that needs to be monitored by setting the PTZ parameters;

Small field of view and large resolution means that by adjusting the PTZ parameters, the target is zoomed to 1/10 to 4/5 of the height of the video image.

Setting the false target feedback feature information database allows targeted screening of targets, which is different from the general calculation mode, can improve the accuracy of the system to capture environmental abnormalities, and has achieved unexpected results. When it is confirmed that the suspected target is a fake target in the secondary target detection, the fake target is described, and the characteristics are fed back to the video image target detection algorithm, which can make the system perform a target detection in this area with a large field of view and a small resolution, and the fake target is merged Features to reduce the probability of false targets being misjudged as suspected targets again. As the running time of the system increases, the false target feature description becomes more and more accurate, the false detection probability of another target detection by the system will be lower and lower, the accuracy will be higher and higher, and the system performance will be automatically improved.

The invention not only ensures that the largest possible area can be monitored, but also the reliability of the monitoring results. After the useful target is detected, the details of the target can be tracked continuously.

Preferably, the present invention provides an intelligent visual perception system, wherein the small field of view and large resolution refers to adjusting the PTZ parameters to zoom the target to 1/6 to 2/3 of the height of the video image and adjust it to the center of the field of view.

Preferably, the present invention provides an intelligent visual perception system, wherein the target feature information includes feature information that characterizes the specific classification of the target and/or feature information that characterizes the specific identity of the target; the feature information that characterizes the specific classification of the target includes humans, animals, One or more of vehicles and/or models, flying objects, and designated foreign objects, where the designated foreign objects include one or more of natural falling objects and/or diffuse objects, and human remains; those that characterize the specific identity of the target The characteristic information includes one or more of the identity of the person, the type of the animal, the license plate of the car, and the type of other foreign objects. In real life, the intelligent visual perception system is applied in the security field. By establishing intelligent recognition algorithms and feature libraries, for suspected targets in the monitoring field, when the suspected target is detected as a true target, the specific classification information and target of the target are represented The specific identity information is sent to the monitoring terminal to make the alarm information true, specific, and credible, which is beneficial to the operator to take reasonable measures to eliminate the intrusion target in the follow-up, and to improve the efficiency of emergency response.

Preferably, the intelligent visual perception system provided by the present invention sets an update rate for the false target feature description information, and optimizes the false target feedback feature information database. The present invention sets the update rate for the feature description information of the false target, so that the false target feedback feature information database has time characteristics, which is more realistic, accelerates the self-adaptation and self-learning speed of the feature database, and further improves the accuracy of one-time detection of the system. Performance, thereby improving the overall performance of the system.

Preferably, the present invention provides an intelligent visual perception system, wherein there are N monitoring areas, where N≥1; the system monitoring process includes:

Perform a target detection in the M-th monitoring area. When a target detection is found to be undoubtedly a target, adjust the camera's field of view direction and focal length, and perform a target detection on the next monitoring area with a large field of view and small resolution, where 1≤ M≤N;

Track the real target in the M-th monitoring area. When the real target disappears or the tracking time reaches the set value, adjust the camera's field of view direction and angle of view, and use a small field of view and large resolution to monitor the next one in the M-th monitoring area. Real target tracking;

When all real targets in the M-th monitoring area disappear or the tracking time reaches the set value, adjust the camera's field of view direction and focal length, and perform a target detection on the next monitoring area with a large field of view and small resolution;

Perform target detection on N monitoring areas in turn, cyclically.

In the present invention, the inventor improves the detection efficiency of the monitoring area by setting the target tracking conditions, such as whether a suspected target is found, the target disappearance time, and the tracking time setting value, etc., and realizes the sequential detection of true targets in the area. And tracking, as well as the sequential detection of multiple monitoring areas, further improve the detection area size and detection distance of a single device, thereby improving equipment utilization, reducing equipment costs and floor space, and having good economic benefits.

Preferably, the present invention provides an intelligent visual perception system, in which a target detection is performed on all N monitoring areas respectively, and the respective feature weight of each monitoring area is calculated for the area where a suspect target is found in a target detection; according to each monitoring area The size of the respective feature weights is used to perform secondary target detection and tracking for each monitoring area in the specified order.

In the present invention, by calculating the weight of each monitoring area, the inventor can calculate the occurrence probability of each monitoring area target and other attributes, ensuring that the system can prioritize the monitoring of areas with relatively large feature weights, which is beneficial to hierarchical early warning and enables operators Prioritize the processing of more suspected targets, or the target moving speed is relatively fast, or other focused monitoring areas in the specific monitoring field where the characteristics of the target are concentrated, and prioritize the elimination of potential safety hazards in this area, thereby reducing the alarm response time of the entire security system . In practical applications, such as in the railway system, this setting of the priority of regional monitoring can enable railway workers to prioritize the removal of important obstacles affecting the passage of trains on the track and its surroundings, such as a large number of sheep passing by, and road collapse covering the railway track. It won time for incident intervention and system linkage, thus saving most of the casualties and hundreds of millions of property losses.

Preferably, the present invention provides an intelligent visual perception system, wherein the system includes one or more intelligent visual perception devices, and the intelligent visual perception devices include:

Camera, used for video image acquisition, including one or more of focus motor, zoom motor, drive module, image signal acquisition and processing unit;

Transmission mechanism, used to adjust the direction and size of the camera's field of view;

The data processing unit is used to analyze and process the video data collected by the camera, control the camera to adjust the focal length, control the transmission mechanism to adjust the direction and size of the camera's field of view, and/or exchange information with the cloud platform or data center;

The communication interface unit is used for information interaction with the cloud platform and/or data center, and/or linkage information interaction with other sensing or action devices on site, and/or other related systems, including wired and/or wireless interfaces;

The power management unit is used to supply power to all power-consuming units;

The protective shell is used to encapsulate each unit and play a protective role.

Preferably, the present invention provides an intelligent visual perception system, wherein the intelligent visual perception device further includes a light unit for supplementing light to the area monitored by the camera; there are one or more light units, including visible light sources and infrared light sources. One or more of the light sources; the transmission mechanism adjusts the direction and size of the field of view of the light unit.

In the present invention, video image acquisition and tracking can be achieved through camera focusing, illumination unit fill light, transmission mechanism to drive the angle adjustment of the camera and illumination power supply; suspected target discovery, true and false target judgment, false target feature description, etc., need The data processing unit implements the video image target detection algorithm to achieve this. The intelligent visual perception device provided by the present invention, through the combination of the above hardware and software, can realize primary target detection and secondary target detection in the monitoring area, distinguish whether the suspected target is a true target or a false target; describe the characteristics of the false target, and Real target tracking, etc., is achievable and technologically advanced. The intelligent visual perception system of the present invention includes one or more of the above-mentioned intelligent visual perception devices, which is beneficial to expand the monitoring area, covers a relatively large area and relatively high security level monitoring scene, and provides a full range of safety warnings.

Preferably, the present invention provides an intelligent visual perception system, wherein the system includes one or more intelligent visual perception devices, in the intelligent visual perception device, there are one or more cameras, and the cameras are visible light cameras and/or infrared cameras , Infrared cameras include near-infrared cameras and/or infrared thermal imaging cameras.

Preferably, one or more cameras complete the video image acquisition in the primary target detection, and/or the video image acquisition in the secondary target detection, and/or the video image acquisition in the real target tracking.

Preferably, the infrared camera completes the video image acquisition in the primary target detection; the visible light camera completes the video image acquisition in the secondary target detection, and/or the video image acquisition in the real target tracking.

Preferably, a visible light camera completes the video image acquisition in the primary target detection, and/or the video image acquisition in the secondary target detection, and/or the video image acquisition in the real target tracking.

The infrared thermal imaging camera has the characteristics of working normally in the outdoor natural environment of -40℃～+70℃ (direct sunlight), can penetrate smoke, fog, and haze, has high image clarity, can night vision and is sensitive to the temperature of the shooting object, etc. , Used for a target detection, it is sensitive to the identification of moving objects that can usually generate early warning signals, especially living things. The visible light camera has the advantages of stable performance and high camera clarity. The infrared thermal imaging camera performs a target detection and finds the suspected target. The visible light camera is used for secondary target detection, and the suspected target is amplified to realize the judgment of the true target and the false target. , Can meet the requirements of system stability and practicability under harsh environmental conditions, and meet the needs of accurate system judgment.

Preferably, the transmission mechanism is used to adjust the horizontal and/or vertical field of view direction and size of the camera and/or the light unit, including a drive motor, a horizontal shaft, and a vertical shaft; the drive motor drives the camera and/or the light unit around the shaft The horizontal rotation is 0-360 degrees, and the vertical rotation is 0-180 degrees.

The transmission mechanism, by driving the camera and the light unit to rotate at a certain angle around the horizontal and vertical shafts, meets the requirements of the camera to collect video images to cover the area that needs to be monitored, and realizes that in the prior art with a simple device structure, multiple The camera can cooperate with each other to realize the image collection needs, and the camera angle can be stably controlled, the stability of the image collection during real target tracking can be realized, and the object of the present invention can be further achieved.

Preferably, the data processing unit adopts a chip with video image processing capabilities, integrates image target detection algorithm programs, and performs real-time video image processing; when the suspected target is identified as a false target, the false target is characterized, and the characteristics are fed back Give the video image target detection algorithm program.

In the present invention, it is necessary to collect video images in the monitoring area during a target detection to determine whether there is a suspected target. The judgment process needs to compare the collected video images with the feature library. If the system judges that there is a suspected target, and after the second target detection, the suspected target is found to be a false target, it means that the features of the false target are not included in the feature database, which leads to additional actions of the system. Therefore, the false target is characterized and fed back to the video image target detection algorithm, which is conducive to the rapid identification of specific obstacles that may appear or non-obstacles that appear regularly in a specific monitoring field. During a target detection, Exclude the range of suspected targets, save the time of image collection and judgment of the system, thereby reducing the time for the warning information feedback of the true target. In certain scenarios, it is more conducive to the staff to eliminate the obstacles and avoid the occurrence of accidents, which has strong practical significance. .

Preferably, the communication interface unit includes an input interface and an output interface; the input interface is used to receive external device signals; the output interface is used to send signals collected or received by the system, and the connection mode includes wireless and/or wired modes; among them, the wireless mode Including one or more of WIFI, BT, ZIGBEE, LORA, 2G, 3G, 4G, 5G, NB-IOT; wired methods include AI/AO, DI/DO, RS485, RS422, RS232, CAN bus, LAN, One or more of optical fibers.

Preferably, the present invention provides an intelligent visual perception system, wherein the intelligent visual perception device receives signals from other sensing or motion devices in the monitoring area through an input interface, and when the sensing or motion device sends abnormal situation signals and/ Or when there is an alarm message, the intelligent visual perception device adjusts the direction and focal length of the camera's field of view, and prioritizes target detection in the area where the sensor or action device is located.

The intelligent visual perception device provided by the present invention can receive the sensor equipment signal in the monitoring area, process the dangerous signal recognized by the external sensor, give priority to the target detection in the area, and realize the secondary verification and target recognition of the information sent with the external sensor , Alarm and characteristic information sending. This linkage mechanism truly realizes the interconnection of sensor devices, saves the system's response time to external early warning events, and at the same time makes up for the fact that external sensors do not understand the specific characteristics of the source of danger, and cannot accurately reflect the alarm information received, even The real problem of being at a loss as to what to do, and wasting the precious saving time before the accident, so as to truly play the role of safety early warning of this system.

Preferably, the present invention provides an intelligent visual perception system, wherein the intelligent visual perception device provides its perception data and/or result information to other sensing or action devices in the monitoring area through an output interface for data fusion and / Or joint judgment, and / or direct control or linkage of sensing or action devices, that is, system linkage.

Preferably, the present invention provides an intelligent visual perception system, wherein other sensing or action devices in the monitoring area include alarm sound and light equipment, access control equipment, fire fighting equipment, obstacle removal equipment, animal repelling equipment, trailer equipment, and cleaning equipment. Equipment, inspection equipment, impulse mitigation equipment, emergency stop equipment, shunt equipment, on-site and/or external communication equipment, explosion-proof equipment, medical rescue equipment, shelter equipment, drone transportation equipment, personnel evacuation and/or safety evacuation equipment One or more of the information; the information includes alarm information and/or target characteristic information.

The intelligent visual perception device provided by the present invention can provide sensory data output to other systems or equipment in the monitoring area. These systems or devices can perform data fusion and joint judgment on the data signals output by the intelligent visual perception device, and can also directly control other local devices. Since the intelligent visual perception device of the present invention can directly control external sensing equipment or system linkage, when there is a hidden accident or emergency danger in the monitoring area, it not only sends alarm information and/or target feature information to the monitoring terminal, but also It can also control or link alarm sound and light equipment, access control equipment, fire fighting equipment, obstacle removal equipment, etc., to eliminate accidental hazards or sources of danger in a targeted manner, and perform emergency braking and on-site diversion of equipment that is operating closely. Measures such as establishing contact between the communication equipment and the monitoring area, reduce the external response time to the alarm information sent by the system, seize the golden gap before the accident, and realize the emergency avoidance of personnel and transportation devices in the monitoring area. This will avoid accidents to the greatest extent, reduce accident damage, and protect the safety of personnel and property, which is of great practical significance.

Preferably, the present invention provides an intelligent visual perception system, wherein the power management unit includes a battery integrated inside the system, and/or an external solar panel, and/or a wired power supply.

Preferably, the protective housing includes an interface board, a window, and a fixing seat;

There are one or more interfaces on the interface board to connect with external units;

The window adopts light-transmitting material to respectively transmit the video image collected by the camera and/or the light emitted by the light unit;

The fixing seat is used to fix the protective shell on the external bracket.

The protective shell design of the intelligent visual perception device provided by the present invention can make the camera, lighting unit, data processing unit, transmission mechanism, communication interface unit, power management unit, etc. of the device be protected by a solid shell. The protective shell is provided with windows for projecting video images and light emitted by the lighting unit, using light-transmitting materials, etc., so that the camera and the lighting unit do not directly contact the external environment, thereby improving the stability and service life of the system. Different numbers of window designs can realize the specific image acquisition needs of the present invention under different application conditions.

Preferably, the present invention provides an intelligent visual perception system, wherein the alarm information and/or target characteristic information are output to the monitoring terminal, and/or the alarm information and/or target characteristic information is output to the data center and/or cloud platform, Start the alarm processing service, distribute information to the monitoring terminal, and complete one or more functions including alarm handling and/or intervention and/or system linkage.

Preferably, the present invention provides an intelligent visual perception system, wherein the cloud platform and/or data center includes a server and software for image recognition through image target detection algorithms; monitoring terminals are one or more for display Target recognition result information, receiving alarm information, remote configuration and control, including intelligent terminal equipment and its running management software, intelligent terminal equipment includes but not limited to one or more of computers and handhelds.

The alarm information, target feature information, etc. sent by the intelligent visual recognition system of the present invention can be sent to the monitoring terminal through the cloud platform and/or data center, ensuring the stability of data transmission and the remote control of the system. At the same time, in the data center, combined with other feature databases, it is possible to further identify the true target, including identity information, so as to issue more accurate alarm information and feature information reports. This solution makes up for the shortcomings in the prior art that in the security field, the monitoring equipment sends out fuzzy information, which causes the monitoring terminal to be unable to eliminate the obstacles and eliminate the danger of accidents in time after receiving the information.

Preferably, the present invention provides an intelligent visual perception system, wherein the server includes a virtual server, including one or more of a local server, an edge cloud, and a public cloud.

The intelligent visual perception system provided by the present invention can find targets, track targets, recognize targets and generate alarm information and target feature recognition information in real time, eliminate non-suspected targets through one target detection, and distinguish between suspected targets through secondary target detection. True and false targets, and feature descriptions of false targets to supplement the feature library, reduce the rate of misjudgment of a target detection, and continue to track true targets. The present invention realizes the intelligent capture, discrimination and feature description of environmental abnormalities, and solves the problem of non-intelligent monitoring of the environment in the security field in the prior art. The operator needs to keep an eye on the screen, and artificially judge whether there is environmental hazard in real time. Resolution, distinguish the actual dangers in the distant place, there is a great possibility of misjudgment, which affects the emergency response time, which delays the timing of accident rescue, and may cause hundreds of millions of property losses and many practical problems of casualties. Has great social value.

Other features and advantages of the present invention will be described in the following description, and partly become obvious from the description, or understood by implementing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the structures indicated in the specification, claims and drawings.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Figure 1 shows the work flow of the intelligent visual perception system provided by the present invention;

Fig. 2 shows the working process of target detection and/or identification for multiple monitoring areas provided by the present invention;

FIG. 3 shows a workflow of performing target detection and/or recognition in sequence by setting area weights provided by the present invention;

Figure 4 shows the data flow of the intelligent visual perception system provided by the present invention;

Figure 5 shows a single-camera intelligent visual perception device provided by the present invention;

Figure 6 shows a single-camera, single-light source intelligent visual perception device provided by the present invention;

Figure 7 shows a dual-camera, dual-light source intelligent visual perception device provided by the present invention;

Figure 8 shows a dual-camera, single-light source intelligent visual perception device provided by the present invention;

Figure 9 shows that the system provided by the present invention finds that there is a suspected target in the distance after a target detection in a railway scene;

FIG. 10 shows that the system provided by the present invention finds a true target through secondary target detection in a railway scene;

Figure 11 shows that the system provided by the present invention tracks real targets in a railway scene.

detailed description

In order to further clarify the present invention, a series of examples are given below. It should be pointed out that these examples are completely illustrative. The purpose of these embodiments is to fully clarify the meaning and content of the present invention, but not to limit the present invention to the scope of the embodiments.

First of all, it should be noted that the present invention is an application of computer technology in the field of security and protection. During the implementation of the present invention, the application of multiple software function modules will be involved. The applicant believes that after carefully reading the application documents, accurately understanding the realization principle and purpose of the invention, and combining the existing known technologies, those skilled in the art can fully use the software programming skills they have mastered to realize the invention. .

Example 1:

The first implementation of an intelligent visual perception system:

The intelligent visual perception system includes a variable focal length camera and a false target feedback feature information library in a designated monitoring area. First, establish a false target feedback feature information database in a specific monitoring area, including false target location information and false target feature description information. Perform video image collection on the monitoring area at the first resolution. Specifically, as shown in Figure 1, the system adjusts the camera's field of view direction and focal length, sets the PTZ parameters, so that the video image covers the preset monitoring area that needs to be monitored, and the large field of view and small resolution perform video image collection on the designated monitoring area . A target detection algorithm is used to perform a target detection in the area on the video image collected by the camera. Perform a target detection in the area, including: judging whether the designated monitoring area contains suspected targets based on the false target feedback feature information database. Further, based on the false target feedback feature information database, it is determined whether the image features in the monitoring area meet the specified conditions with the false target feedback feature description and location information matching degree of the area. If it is satisfied, it is not judged as a suspected target. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, for example, when the matching degree reaches 90%, it is considered that the area has a high matching degree with the false target feedback feature description. The object at the corresponding location of the target feedback feature description is not a suspected target, so the area will be classified as a background with a high probability, that is, when the corresponding location of the false target feedback feature description in the area matches the specified value, it will not be classified Classified as a suspected target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

When a target detection is found to be undoubtedly a target, adjust the camera's field of view direction and focal length, use a large field of view and small resolution to capture the video image of the next designated monitoring area, that is, the preset monitoring area, and use a target detection algorithm to control the camera. The collected video images are subjected to a target detection in the area.

When a suspected target is found in a target detection, by adjusting the PTZ parameters, adjusting the direction and focal length of the camera's field of view, zooming the target height, the suspected target is captured with a second resolution, and the second resolution is greater than the first resolution. In the embodiment of the present invention, the suspected targets in the monitoring area are collected sequentially with a small field of view and large resolution, and a secondary target detection algorithm is adopted to perform secondary target detection on the collected video images to distinguish the suspected targets as true targets. Or false goals. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

When the secondary target is detected to be a true target, the alarm information is output to the monitoring terminal, and target characteristic information is generated. Adjust the direction and focal length of the camera's field of view, and continue to track the target for a period of time. At the same time, the target feature information generated by the secondary target detection is sent to the monitoring terminal.

Target feature information includes feature information that characterizes the specific classification of the target, including people, animals, cars and/or models, flying objects, designated foreign objects, that is, other foreign objects that should not appear, including natural falling objects and/or diffuse objects, such as falling rocks One or more of, mudslides, and human remains, or characteristic information that characterizes the specific identity of the target, including one or more of the identity of the person, the type of animal, and the license plate of the vehicle. The specific information types and contents are defined according to the specific application environment.

Track the real target. When the real target disappears or the tracking time reaches the set value, adjust the direction and size of the camera's field of view to track the next real target with a small field of view and large resolution.

When the secondary target detection finds that the suspected target of the primary target detection is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image to obtain the position information of the false target, and extract the false target feature description information. Update the false target feedback feature information database to reduce the false detection probability of a target detection in the system. As the running time of the system increases, the false target feature description becomes more and more accurate, the false detection probability of another target detection by the system will be lower and lower, the accuracy will be higher and higher, and the system performance will be automatically improved.

When performing secondary target detection and tracking, when all true targets disappear or the tracking time reaches the set value, adjust the camera's field of view direction and focal length, and perform target recognition on the next preset monitoring area with a large field of view and small resolution. The method is the same as the target recognition method in the first preset monitoring area.

By analogy, as shown in Figure 2, target detection and/or recognition are performed on N (0<N) preset monitoring areas, and finally, target detection and/or recognition are restarted from the first preset monitoring area, and the execution is repeated The above steps. In the M-th monitoring area, perform secondary detection on T suspected targets. The secondary detection with large field of view and small resolution means that the video image covers the area that needs to be monitored by setting the PTZ parameters; then The next area, that is, the M+1 monitoring area, performs a second inspection.

Small field of view and large resolution means that by adjusting the PTZ parameters, the target height is zoomed to 1/10 to 4/5 of the video image height, and adjusted to the center of the field of view.

Example 2:

On the basis of the foregoing embodiment, a second implementation manner of an intelligent visual perception system is also provided:

Establish a false target feedback feature information database in a specific monitoring area, including false target location information and false target feature description information. As shown in Figure 1, the system adjusts the direction of the camera's field of view, sets the PTZ parameters so that the video image covers the area that needs to be monitored, and collects video images in the first preset monitoring area with the large field of view and small resolution. A target detection algorithm is used for the video image collected by the camera to perform a target detection in the first preset monitoring area. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, the area will be classified as a background with a high probability, and it will not be classified as a suspect Target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

When a target detection is found to be undoubtedly a target, the intelligent visual perception device continues to collect video images of the next preset monitoring area with a large field of view and small resolution. A target detection algorithm is used to perform a target in the area on the collected video image. Detection.

When a suspected target is found in a target detection, the first preset monitoring area feature weight Q1 is given; at the same time, the system continues to collect video images of other preset monitoring areas with a large field of view and small resolution, and uses a secondary target detection algorithm , Perform a target detection in the monitoring area on the collected video images. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

Exemplarily, there are N monitoring areas, where N≥1; the system monitoring process includes:

Perform target detection on N monitoring areas in turn, cyclically.

Further, as shown in Fig. 3, a target detection is performed on all N monitoring areas respectively, and the respective feature weight of each monitoring area is calculated for the area where a suspected target is found in a target detection; according to the size of the respective feature weight of each monitoring area, according to Perform secondary target detection and tracking on each monitoring area in the specified order. Exemplarily, a target detection of suspected targets is performed on N (0<N) preset monitoring areas in sequence, the area where the suspected target exists is found, and the feature weight Qi is given to the area where the suspected target exists. According to the size of the regional feature weight Qi, reorder the monitoring areas with suspected targets. The largest weight is the first area, and its weight is Q1, and the smallest weight is the W (W<=N) area, and its weight is Qw .

The system adjusts the camera's field of view direction and focal length, adjusts the PTZ parameters, zooms the target height, and collects the video image of the suspected target in the first area where the suspected target exists with a small field of view and large resolution, and passes the video image target detection algorithm to The collected video images are subjected to secondary target detection.

When the secondary target detection finds that the suspected target of the primary target detection is a true target, the first alarm information is output, and the first target characteristic information is generated. Adjust the direction and focal length of the camera's field of view, and continue to track the target for a period of time. At the same time, the first target feature information generated by the secondary target detection is sent to the cloud platform or data center, and the cloud platform or data center is based on the target feature sent by the intelligent visual perception device Information 1 passes through the target recognition algorithm to perform target recognition, and sends the obtained second target characteristic information and second alarm information to the monitoring terminal; the monitoring terminal responds to the second alarm information, the first target characteristic information and/or the second target Characteristic information is processed and displayed.

The first target feature information includes feature information that characterizes the specific classification of the target, including people, animals, cars and/or models, flying objects, other foreign objects that should not appear, including natural falling objects and/or diffuse objects, such as falling rocks, mudslides , And one or more of human remains. The second target characteristic information includes characteristic information that characterizes the specific identity of the target, including one or more of the identity of the person, the type of animal, the license plate of the car, and the type of other foreign objects. The specific information types and contents are defined according to the specific application environment.

When the secondary target detection finds that the suspected target of the primary target detection is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image, the false target feature description information is extracted, and the false target feature description information is set Determine the update rate, optimize the false target feedback feature information database, and further reduce the probability of false detection of a target detection in the system.

When performing secondary target detection and tracking on the first area, when all true targets disappear or the tracking time reaches the set value, adjust the camera's field of view direction and focal length, and target the second area where there are suspected targets with a small field of view and large resolution The detection and/or recognition method is the same as the detection and/or recognition method for the first area target where there is a suspected target.

By analogy, the above steps are repeated to perform secondary target detection and/or recognition for all W regions with suspected targets.

Finally, the target detection and/or recognition is performed again on the first preset monitoring area, and the above steps are automatically executed, and the target detection and/or recognition is performed on the N areas in a loop.

Among them, the large field of view and small resolution means that the video image covers the area that needs to be monitored by setting the PTZ parameters;

Small field of view and large resolution means that by adjusting the PTZ parameters, the target height is zoomed to 1/6 to 2/3 of the video image height, and adjusted to the center of the field of view.

Example 3:

On the basis of the foregoing embodiment, a third implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. A structure of the intelligent visual perception device is composed of a visible light camera, a transmission mechanism, a data processing unit, a communication interface unit, a power management unit, and a protective casing. The shell is left with a window and power and signal line interfaces. The window is sealed with a light-permeable material for the camera to collect video images. The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera through the control commands sent by the data processing unit, including a drive motor, a horizontal shaft, a vertical shaft, and a control line. The drive motor drives the camera to rotate 0-360 degrees horizontally around the axis of rotation, and 0-180 degrees up and down.

The communication interface unit mainly includes wired and wireless communication interfaces for receiving external device signals and sending signals collected or received by the system. Its connection methods include wireless and/or wired methods; among them, wireless methods include WIFI, BT, ZIGBEE, LORA One or more of, 2G, 3G, 4G, 5G, NB-IOT; wired methods include one or more of AI/AO, DI/DO, RS485, RS422, RS232, CAN bus, LAN, and optical fiber .

The data processing unit is used to analyze and process the video data collected by the camera, control the camera to adjust the focal length, control the transmission mechanism to adjust the camera angle, and/or exchange information with the cloud platform or data center. The power management unit is mainly used to supply power to the entire intelligent video sensing device. The transmission mechanism controls the left and right swing positions of the camera.

As shown in FIG. 5, the intelligent visual perception device includes a visible light camera 501, a transmission mechanism 502, a data processing unit 503, a communication interface unit 504, a power management unit 505, and a protective housing 507.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the fixing seat is used to fix the protective shell and can be fixed on the external bracket. In this embodiment, the housing has a window 508 and a power and signal line interface board 510. The window 508 is sealed with a light-permeable material for video image capture by the camera. The transmission mechanism 502 adjusts the up, down, left, and right field of view directions of the camera 501 by receiving control commands sent by the data processing unit 503. The communication interface unit 504 mainly includes wired and wireless communication interfaces. The power management unit 505 is mainly used to supply power to the entire intelligent video perception device, and in this embodiment is a battery inside the system. Among them, the camera is used for video image acquisition, including a focus motor, a zoom motor, a drive module, an image signal acquisition and processing unit, and so on.

The system workflow is as follows:

(1) According to the monitoring range of the device, the number and position of the monitoring area are preset, and the PTZ parameter of each area is set with the maximum magnification that can cover the area. Establish a false target feedback feature information database in a specific monitoring area, including false target location information and false target feature description information.

(2) For the first preset monitoring area, set the PTZ parameters, perform video image collection with the large field of view and small resolution, and transmit the collected video images to the data processing unit 503 in real time.

(3) The data processing unit 503 performs a target detection on the collected video image through a target detection algorithm. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, the area will be classified as a background with a high probability, and it will not be classified as a suspect Target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 503 sends a control command to the camera 501 and the transmission mechanism 502, and the transmission mechanism 502 adjusts the direction of the camera 501's field of view , The camera 501 field of view direction is aligned with the second preset monitoring area, and the focal length of the camera 501 is adjusted to collect video images in this area with a large field of view and small resolution. The data processing unit 503 performs a video image target detection algorithm on the collected The video image performs a target detection of the target.

(5) When a suspected target is found in a target detection, adjust the PTZ parameters: the data processing unit 503 sends a control command to the camera 501 and the transmission mechanism 502, and the transmission mechanism 502 adjusts the field of view direction of the camera 501 so that the field of view direction of the camera 501 is aligned For the suspected target, adjust the focus of the camera 501. By adjusting the PTZ parameters, the height of the suspected target is scaled to 1/3 of the height of the video image, and the video image of the suspected target is collected with this small field of view and large resolution. The data processing unit 503 performs a secondary target detection algorithm on the collected video image. Perform secondary target detection. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(6) When the secondary target detection determines that it is a true target, the data processing unit 503 outputs alarm information and generates the true target characteristic information. At the same time, the data processing unit 503 sends control commands to the camera 501 and the transmission mechanism 502, the transmission mechanism 502 adjusts the direction of the field of view of the camera 501, adjusts the focal length of the camera 501 in real time, and tracks the real target detected in real time. At the same time, the data processing unit 503 sends the characteristic information and alarm information of the real target to the monitoring terminal.

Target feature information includes feature information that characterizes the specific classification of the target, including people, animals, cars and/or models, flying objects, other foreign objects that should not appear, including natural falling objects and/or diffuse objects, such as falling rocks, mudslides, and One or more of human remains, etc., or characteristic information that characterizes the specific identity of the target, including one or more of the identity of the person, the type of animal, the license plate of the car, and the type of other foreign objects. The specific information types and contents are defined according to the specific application environment.

(7) When the camera 501 is tracking a real target, when the real target disappears or the tracking time reaches the set value, the data processing unit 503 sends a control command to the camera 501 and the transmission mechanism 502, and the transmission mechanism 502 adjusts the direction of the field of view of the camera 501 so that The direction of the field of view of the camera 501 is aimed at the next real target for tracking.

(8) When the secondary target detection determines that it is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 503 characterizes the false target and extracts the false target feature description Information, update the false target feedback feature information database to reduce the probability of a target detection error after the algorithm. As the running time of the system increases, the false target feature description becomes more and more accurate, the false detection probability of another target detection by the system will be lower and lower, the accuracy will be higher and higher, and the system performance will be automatically improved.

The comparative example is an image acquisition, and the YOLO V3 target classification detection algorithm based on the deep neural network is used to perform a target detection.

It has been verified by experiments that with the increase of the operating cycle, the accuracy of the system's one-time target detection is as follows:

(9) When the camera 501 is tracking a real target, when all real targets disappear or the tracking time reaches the set value, the PTZ parameters are set for the second preset monitoring area, and the video image is collected with the large field of view and small resolution , Transmit the collected video images to the data processing unit 503 in real time, and the data processing unit 503 performs a target detection on the collected video images through a video image target detection algorithm.

(10) By analogy, follow the steps (2) to (9) to perform target detection and/or recognition on N preset areas, and finally, restart target detection and/or recognition from the first preset monitoring area, and automatically loop.

Example 4:

On the basis of the foregoing embodiment, a fourth implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. A structure of the intelligent visual perception device is composed of a visible light camera, a transmission mechanism, a data processing unit, a communication interface unit, a power management unit, and a protective shell. The shell is left with a window and power and signal line interfaces. The window is sealed with a light-permeable material for the camera to collect video images. The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera through the control commands sent by the data processing unit, including a drive motor, a horizontal shaft, a vertical shaft, and a control line. The drive motor drives the camera to rotate 0-360 degrees horizontally around the axis of rotation, and 0-180 degrees up and down.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the fixing seat is used to fix the protective shell and is fixed on the external bracket. In this embodiment, the housing has a window 508 and a power and signal line interface board 510. The window 508 is sealed with a light-permeable material for video image capture by the camera. The transmission mechanism 502 adjusts the up, down, left, and right field of view directions of the camera 501 by receiving control commands sent by the data processing unit 503. The communication interface unit 504 mainly includes wired and wireless communication interfaces. The power management unit 505 is mainly used to supply power to the entire intelligent video perception device, and in this embodiment is a battery inside the system. Among them, the camera is used for video image acquisition, including a focus motor, a zoom motor, a drive module, an image signal acquisition and processing unit, and so on.

The system workflow is as follows:

(5) When a suspected target is found in a target detection, the data processing unit 503 gives the first preset monitoring area feature weight Q1, which is obtained from the number of suspected targets in the area; set the PTZ parameter to a large field of view and a small resolution Cover the second preset monitoring area, perform video image collection on this area, transmit the collected video image to the data processing unit 503 in real time, and perform a target detection of the target through the video image target detection algorithm.

(6) By analogy, repeat the above steps (1) ~ (5), perform a target detection on N monitoring areas, distinguish W areas where there are suspected targets, and renew the W areas according to the size of the area feature weight Qi Sorting, the largest weight is the first area, and the smallest weight is the W-th area.

In this embodiment, each area feature weight Qi (0<i≤N) is obtained from the number of suspected targets in each area.

(7) The intelligent visual perception device adjusts the direction and focal length of the field of view of the camera 501. By adjusting the PTZ parameters, the height of the suspected target is zoomed to 1/2 of the height of the field of view, and the first area with the suspected target is performed with the small field of view and large resolution. Video image collection, real-time transmission of the collected video images to the data processing unit 503, the data processing unit 503 through the secondary target detection algorithm, on the collected video images in order to perform secondary target detection of suspected targets in the area. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(8) When the secondary target detection determines that the suspected target is a true target, the data processing unit generates target feature information 1 and warning information 1 of the true target. The data processing unit 503 sends control commands to the camera 501 and the transmission mechanism 502. The transmission mechanism 502 adjusts the field of view direction of the camera 501, adjusts the focal length of the camera 501 in real time, and tracks the real target detected in real time; The target characteristic information 1 of the detected real target is sent to the cloud platform or data center; the cloud platform or data center is based on the target characteristic information 1 sent by the intelligent visual perception device through the target recognition algorithm, and the target is identified, and the target characteristic information is obtained 2 And alarm information 2 are sent to the monitoring terminal; the monitoring terminal processes and displays the alarm information 1, the alarm information 2, the target characteristic information 1 and/or the target characteristic information 2.

Target feature information 1 includes feature information that characterizes the specific classification of the target, including people, animals, cars and/or models, flying objects, other foreign objects that should not appear, including natural falling objects and/or diffuse objects, such as falling rocks, mudslides, And one or more of human remains. The target characteristic information 2 includes characteristic information that characterizes the specific identity of the target, including one or more of the identity of a person, the type of animal, the license plate of the car, and the type of other foreign objects. The specific information types and contents are defined according to the specific application environment.

(9) When the camera 501 is tracking a real target, when the target disappears or the tracking time reaches a set value, the data processing unit 503 sends a control command to the camera 501 and the transmission mechanism 502, and the transmission mechanism 502 adjusts the direction of the camera 501's field of view. Make the field of view of the camera 501 align with the next real target, adjust the focal length of the camera 501, collect video images of the next real target with a small field of view and large resolution, and transmit the collected video images to the data processing unit 503 in real time. 503 continues to perform secondary target detection on the collected video images through the video image target detection algorithm.

(10) When the secondary target detection determines that the suspected target is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 503 characterizes the false target and extracts the false target Feature description information, set the update rate for the false target feature description information, optimize the false target feedback feature information database, and further improve the accuracy of a detection algorithm.

(11) The camera 501 tracks the real target. When all real targets disappear or the tracking time reaches the set value, the data processing unit 503 sends a control command to the camera 501 and the transmission mechanism 502, and the transmission mechanism 502 adjusts the direction of the camera 501's field of view. Align the field of view of the camera 501 with the second area, adjust the focal length of the camera 501, collect video images in the second area with a small field of view and large resolution, and transmit the collected video images to the data processing unit 503 in real time to perform the secondary target Target detection, the detection method is the same as the second detection method of the first area target.

(12) By analogy, follow steps (7) to (11) to perform secondary target detection and/or recognition for W areas with suspected targets, and finally, start again from step (2) and start from the first preset monitoring The area starts to recognize the target, and it loops automatically.

(13) Multiple intelligent visual perception devices form an intelligent visual perception system. By setting different monitoring ranges for each intelligent visual perception device, the above-mentioned target detection and/or recognition are performed on the preset monitoring areas within each monitoring range to achieve Target monitoring of a wider area.

Example 5:

On the basis of the foregoing embodiment, a fifth implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. A structure of the intelligent visual perception device is composed of a visible light camera, a transmission mechanism, a data processing unit, a communication interface unit, a power management unit, a lighting unit, and a protective housing. There are 2 windows and power and signal wire interfaces in the shell. The 2 windows are sealed with light-permeable materials. One of the windows is used for video image collection by the camera, and the other window is used for the lighting unit to fill light. The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera and the illumination unit through the control commands sent by the data processing unit, including a drive motor, a horizontal shaft, a vertical shaft, and a control line. The driving motor drives the camera and the light unit to rotate 0-360 degrees horizontally around the rotating shaft, and 0-180 degrees up and down.

The data processing unit is used to analyze and process the video data collected by the camera, control the camera to adjust the focus, control the transmission mechanism to adjust the direction and size of the field of view of the camera and/or the light unit, and/or exchange information with the cloud platform or data center . The power management unit is mainly used to supply power to the entire intelligent video sensing device. The light unit mainly includes a light-emitting device, light intensity, and field of view direction range adjustment unit. The light unit and the camera are fixed together, and the transmission mechanism jointly controls the left and right swing positions of the light unit and the camera.

As shown in FIG. 6, the intelligent visual perception device includes a visible light camera 201, a transmission mechanism 202, a data processing unit 203, a communication interface unit 204, a power management unit 205, an illumination unit 206, and a protective housing 207.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the window adopts light-transmitting material, which transmits the video image collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell and is fixed to the external bracket superior. In this embodiment, the shell is left with

windows

208 and 209, as well as the power and signal line interface board 210. The

windows

208 and 209 are sealed with light-permeable materials. The window 208 is used for video image capture by the camera, and the window 209 Used to fill light. The transmission mechanism 202 adjusts the vertical field of view direction of the camera 201 by receiving the control command sent by the data processing unit 203, and adjusts the horizontal field of view direction of the camera 201 through the rotating shaft 211 extending outside the protective housing and mounted on the fixed bracket. The communication interface unit 204 mainly includes wired and wireless communication interfaces. The power management unit 205 is mainly used to supply power to the entire intelligent video perception device, and in this embodiment is a battery inside the system. The illumination unit 206 is a visible light source, and mainly includes a light-emitting device and a unit for adjusting light intensity and light range. The illumination unit 206 and the camera 201 are fixed together, and the transmission mechanism 202 jointly controls the left and right swing positions of the illumination unit and the camera. The transmission mechanism includes a driving motor and a turntable.

Among them, the camera is used for video image acquisition, including a focus motor, a zoom motor, a drive module, an image signal acquisition and processing unit, and so on.

The system workflow is as follows:

(1) According to the monitoring range of the device, the number and location of the monitoring area are preset, and the PTZ parameter of each area is set with the maximum magnification that can cover the area. Establish a false target feedback feature information database in a specific monitoring area, including false target location information and false target feature description information.

(2) For the first preset monitoring area, set the PTZ parameters, perform video image collection with the large field of view and small resolution, and transmit the collected video images to the data processing unit 203 in real time.

(3) The data processing unit 203 performs a target detection on the collected video image through a target detection algorithm. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, the area will be classified as a background with a high probability, and it will not be classified as a suspect Target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 203 sends a control command to the camera 201 and the transmission mechanism 202, and the transmission mechanism 202 adjusts the direction of the camera 201 field of view , The camera 201 field of view direction is aligned with the second preset monitoring area, and the focal length of the camera 201 is adjusted to collect video images in this area with a large field of view and small resolution. The data processing unit 203 performs a video image target detection algorithm on the collected The video image undergoes a target detection.

(5) When a suspected target is found in a target detection, adjust the PTZ parameters: the data processing unit 203 sends a control command to the camera 201 and the transmission mechanism 202, and the transmission mechanism 202 adjusts the direction of the field of view of the camera 201 so that the direction of the field of view of the camera 201 is aligned For the suspected target, adjust the focal length of the camera 201, zoom the height of the suspected target to 1/10 of the height of the video image, and collect the video image of the suspected target with this small field of view and large resolution. The data processing unit 203 performs a secondary target detection algorithm to collect The video image is subjected to secondary target detection. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(6) When the secondary target detection determines that it is a true target, the data processing unit 203 outputs alarm information and generates the true target characteristic information. At the same time, the data processing unit 203 sends a control command to the camera 201 and the transmission mechanism 202. The transmission mechanism 202 adjusts the direction of the field of view of the camera 201, adjusts the focal length of the camera 201 in real time, and tracks the detected real target in real time. At the same time, the data processing unit 203 sends the real target feature information and alarm information to the monitoring terminal.

(7) The camera 201 tracks the real target. When the real target disappears or the tracking time reaches the set value, the data processing unit 203 sends a control command to the camera 201 and the transmission mechanism 202. The transmission mechanism 202 adjusts the direction of the camera 201's field of view so that the camera 201 The direction of the field of view is aligned with the next real target for tracking.

(8) When the secondary target detection determines that it is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 203 characterizes the false target and extracts the false target feature description Information, update the false target feedback feature information database to reduce the probability of a target detection error after the algorithm. As the running time of the system increases, the false target feature description becomes more and more accurate, the false detection probability of another target detection by the system will be lower and lower, the accuracy will be higher and higher, and the system performance will be automatically improved.

(9) The camera 201 tracks the real target. When all real targets disappear or the tracking time reaches the set value, the PTZ parameter is set for the second preset monitoring area, and the video image is collected with the large field of view and small resolution. The collected video images are transmitted to the data processing unit 203 in real time, and the data processing unit 203 performs a target detection on the collected video images through a video image target detection algorithm.

(11) Multiple intelligent visual perception devices form an intelligent visual perception system. By setting different monitoring ranges for each intelligent visual perception device, the above-mentioned target detection and/or recognition are performed on the preset monitoring areas within each monitoring range to achieve Target monitoring of a wider area.

In the work of the intelligent visual perception device, the data processing unit 203 performs real-time analysis on the brightness of the captured video image. When the brightness is insufficient, it sends a control command to the lighting unit 206 in time to adjust the light intensity and light field angle to make the video image captured by the camera 201 The brightness is moderate.

Example 6:

On the basis of the foregoing embodiment, a sixth implementation manner of an intelligent visual perception system is also provided:

The data processing unit is used to analyze and process the video data collected by the camera, control the camera to adjust the focal length, control the transmission mechanism to adjust the angle of the camera and/or the light unit, and/or exchange information with the cloud platform or data center. The power management unit is mainly used to supply power to the entire intelligent video sensing device. The light unit mainly includes a light-emitting device, light intensity, and field of view direction range adjustment unit. The light unit and the camera are fixed together, and the transmission mechanism jointly controls the left and right swing positions of the light unit and the camera.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the window adopts light-transmitting material, which transmits the video image collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell, including fixing on the turntable On, and/or fixed to an external bracket. In this embodiment, the shell is left with

windows

208 and 209, as well as the power and signal line interface board 210. The

windows

208 and 209 are sealed with light-permeable materials. The window 208 is used for video image capture by the camera, and the window 209 It is used to fill light in the lighting unit. The transmission mechanism 202 adjusts the vertical field of view direction of the camera 201 by receiving the control command sent by the data processing unit 203, and adjusts the horizontal field of view direction of the camera 201 through the rotating shaft 211 extending outside the protective housing and mounted on the fixed bracket. The communication interface unit 204 mainly includes wired and wireless communication interfaces. The power management unit 205 is mainly used to supply power to the entire intelligent video perception device, and in this embodiment is an external solar panel. The light unit 206 is visible light, and mainly includes a light-emitting device and a unit for adjusting light intensity and light range. The illumination unit 206 and the camera 201 are fixed together, and the transmission mechanism 202 jointly controls the left and right swing positions of the illumination unit and the camera. The transmission mechanism includes a driving motor and a turntable.

Among them, the camera is used for video image acquisition, including focus motor, zoom motor, drive module, image signal acquisition and processing unit, etc.

The system workflow is as follows:

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 203 sends a control command to the camera 201 and the transmission mechanism 202, and the transmission mechanism 202 adjusts the direction of the camera 201 field of view , The camera 201 field of view direction is aligned with the second preset monitoring area, the focal length of the camera 201 is adjusted, and the video image is collected in this area with a large field of view and small resolution. The data processing unit 203 performs a video image target detection algorithm on the collected video The image performs a target detection of the target.

(5) When a suspected target is found in a target detection, the data processing unit 203 gives the first preset monitoring area feature weight Q1, which is obtained from the number of suspected targets in the area; set the PTZ parameter to a large field of view and a small resolution Cover the second preset monitoring area, perform video image collection on this area, transmit the collected video image to the data processing unit 203 in real time, and perform a target detection of the target through the video image target detection algorithm.

(6) By analogy, repeat the above steps (1) ~ (5), perform a target detection of N monitoring area targets, distinguish W areas where there are suspected targets, and compare W areas according to the size of the area feature weight Qi Re-order, the largest weight is the first area, and the smallest weight is the W-th area.

(7) The intelligent visual perception device adjusts the direction and focal length of the field of view of the camera 201, and by adjusting the PTZ parameters, the height of the suspected target is zoomed to 4/5 of the height of the video image, and the first area where the suspected target exists with the small field of view and large resolution Video image collection is performed, and the collected video images are transmitted to the data processing unit 203 in real time, and the data processing unit 203 performs secondary target detection of suspected targets in the area sequentially on the collected video images through a secondary target detection algorithm. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(8) When the secondary target detection determines that the suspected target is a true target, the data processing unit generates target feature information 1 and warning information 1 of the true target. The data processing unit 203 sends control commands to the camera 201 and the transmission mechanism 202. The transmission mechanism 202 adjusts the field of view direction of the camera 201, adjusts the focal length of the camera 201 in real time, and tracks the real target detected in real time; The target characteristic information 1 of the detected real target is sent to the cloud platform or data center; the cloud platform or data center is based on the target characteristic information 1 sent by the intelligent visual perception device through the target recognition algorithm, and the target is identified, and the target characteristic information is obtained 2 And alarm information 2 are sent to the monitoring terminal; the monitoring terminal processes and displays the alarm information 1, the alarm information 2, the target characteristic information 1 and/or the target characteristic information 2.

(9) When the camera 201 is tracking a real target, when the target disappears or the tracking time reaches the set value, the data processing unit 203 sends a control command to the camera 201 and the transmission mechanism 202, and the transmission mechanism 202 adjusts the direction of the field of view of the camera 201, Make the field of view of the camera 201 aim at the next real target, adjust the focal length of the camera 201, collect video images of the next real target with a small field of view and large resolution, and transmit the collected video images to the data processing unit 203 in real time. 203 After the video image target detection algorithm, continue to perform secondary target detection on the collected video image.

(10) When the secondary target detection determines that the suspected target is a false target, the false target determined in the large-resolution image is mapped to the small-resolution image, and the data processing unit 203 features description of the false target and extracts the false target Feature description information, set the update rate for the false target feature description information, optimize the false target feedback feature information database, and further improve the accuracy of a detection algorithm.

(11) The camera 201 tracks the real target. When all real targets disappear or the tracking time reaches the set value, the data processing unit 203 sends a control command to the camera 201 and the transmission mechanism 202, and the transmission mechanism 202 adjusts the direction of the camera 201's field of view. Align the field of view of the camera 201 with the second area, adjust the focal length of the camera 201, collect video images of the second area with a small field of view and large resolution, and transmit the collected video images to the data processing unit 203 in real time to perform the secondary target Target detection, the detection method is the same as the second detection method of the first area target.

During the work of the entire intelligent visual perception device, the data processing unit 203 analyzes the brightness of the captured video image in real time. When the brightness is insufficient, it sends a control command to the lighting unit 206 in time to adjust the light intensity so that the video image captured by the camera 201 has a moderate brightness. .

Example 7:

On the basis of the foregoing embodiment, a seventh implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. The intelligent visual perception device is composed of 2 visible light cameras, 2 transmission mechanisms, 1 data processing unit, 1 communication interface unit, 1 power management unit, 2 illumination units and 1 protective housing. The shell has 4 windows and power and signal wire interfaces. The 4 windows are sealed with light-permeable materials. Two of the windows are used for the camera to collect video images, and the other two windows are used for the illumination unit to fill light. 2 The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera and the illumination unit through the control commands sent by the data processing unit, including the drive motor, the horizontal shaft, the vertical shaft, and the control line. The driving motor drives the camera and the light unit to rotate 0-360 degrees horizontally around the rotating shaft, and 0-180 degrees up and down.

The data processing unit is used to analyze and process the video data collected by the camera, control the camera to adjust the focal length, control the transmission mechanism to adjust the angle of the camera and/or the light unit, and/or exchange information with the cloud platform or data center. The power management unit is mainly used to supply power to the entire intelligent video sensing device. The light unit mainly includes a light-emitting device, a light intensity, and light range adjustment unit. The light unit and the camera are fixed together, and the transmission mechanism jointly controls the left and right swing positions of the light unit and the camera.

As shown in Figure 7, the intelligent visual perception device includes a visible light camera 301-1, a visible light camera 301-2, a transmission mechanism 302-1, a transmission mechanism 302-2, a data processing unit 303, a communication interface unit 304, a power management unit 305, The light unit 306-1, the light unit 306-2, and the protective shell constitute 307.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the window adopts light-transmitting material, which transmits the video image collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell and is fixed to the external bracket superior. In this embodiment, the shell leaves window 308-1, window 308-2, window 309-1, window 309-2, and power and signal line interface board 310, window 308-1, window 308-2, window 309 -1. The windows 309-2 are all sealed with light-permeable materials. The windows 308-1 and 308-2 are used for the video image collection of the camera 301-1 and the camera 301-2, and the windows 309-1 and 309- 2 is used for the light transmission of the light unit 306-1 and the light unit 306-2, and supplement light for the camera 301-1 and the camera 301-2 to monitor the target.

The transmission mechanism 302-1 adjusts the position of the camera 301-1 left and right and the vertical field of view by receiving the control commands sent by the data processing unit 303, and the transmission mechanism 302-2 adjusts the camera 301-2 by receiving the control commands sent by the data processing unit 303 The position of the field of view from left to right and up and down. The communication interface unit 304 mainly includes wired and wireless communication interfaces. The power management unit 305 is mainly used to supply power to the entire intelligent video sensing device, which is a wired power supply in this embodiment. The illumination units 306-1 and 306-2 mainly include a light emitting device and a light intensity adjustment unit, and the light sources are visible light and/or infrared light, respectively. The light unit 306-1 and the camera 301-1 are fixed together, the light unit 306-2 and the camera 301-2 are fixed together, and the transmission mechanism 302-1 jointly controls the left and right swing positions of the light unit 306-1 and the camera 301-1 , The transmission mechanism 302-2 jointly controls the left and right swing positions of the light unit 306-2 and the camera 301-2. Among them, the camera is used for video image acquisition, including a focus motor, a zoom motor, a drive module, an image signal acquisition and processing unit, and so on.

The system workflow is as follows:

(2) For the first preset monitoring area, set the PTZ parameters, perform video image collection with the large field of view and small resolution, and transmit the collected video images to the data processing unit 303 in real time.

(3) The data processing unit 303 performs a target detection on the collected video image through a target detection algorithm. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, the area will be classified as a background with a high probability, and it will not be classified as a suspect Target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 303 sends a control command to the camera 301-1 and the transmission mechanism 302-1, and the transmission mechanism 302-1 Adjust the field of view direction of the camera 301-1 so that the field of view direction of the camera 301-1 is aligned with the second preset monitoring area. The video image is transmitted to the data processing unit 303 in real time to perform a target detection of the target.

(5) When a suspected target is found in a target detection, adjust the PTZ parameters: the data processing unit 303 sends a control command to the camera 301-2 and the transmission mechanism 302-2, and the transmission mechanism 302-2 adjusts the field of view direction of the camera 301-2. The direction of the field of view of the camera 301-2 is aligned with the suspected target, and the focal length of the camera 301-2 is adjusted. By adjusting the PTZ parameters, the height of the suspected target is scaled to 1/6 of the height of the video image, and the video image of the suspected target is collected with this small field of view and large resolution. The data processing unit 303 performs a secondary target detection algorithm on the collected video image. Perform the secondary target detection of the target. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(6) When the secondary target detection determines that the suspected target is a true target, the data processing unit 303 outputs alarm information 1 and generates the true target characteristic information 1. At the same time, the data processing unit 303 sends control commands to the cameras 301-2 and 301-2. The transmission mechanism 302-2 and the transmission mechanism 302-2 adjust the field of view direction of the camera 301-2, adjust the focal length of the camera 301-2 in real time, and track the real target detected in real time; at the same time, the data processing unit 303 obtains the characteristic information of the real target 1 Send to the cloud platform or data center, and the cloud platform or data center sends the true target characteristic information according to the intelligent visual perception device. 1 After the target recognition algorithm, the target recognition is performed, and the true target characteristic information 2 and alarm information 2 are generated. The true target characteristic information 2 and the alarm information 2 are sent to the monitoring terminal; the monitoring terminal processes and displays the alarm information 1, the alarm information 2, the target characteristic information 1 and/or the target characteristic information 2.

(7) When the camera 301-2 is tracking a real target, when the real target disappears or the tracking time reaches the set value, the data processing unit 303 sends a control command to the camera 301-2 and the transmission mechanism 302-2, and the transmission mechanism 302 -2 Adjust the field of view direction of the camera 301-2 so that the field of view direction of the camera 301-2 is aimed at the next real target, and adjust the focal length of the camera 301-2 to track the next real target with a small field of view and large resolution.

(8) When the secondary target detection determines that it is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 303 performs a feature description on the false target and extracts the feature description of the false target Information, update the false target feedback feature information database to reduce the probability of a target detection error after the algorithm. As the running time of the system increases, the false target feature description becomes more and more accurate, the false detection probability of another target detection by the system will be lower and lower, the accuracy will be higher and higher, and the system performance will be automatically improved.

(9) The camera 301-2 tracks the real target. When all real targets disappear or the tracking time reaches the set value, set the PTZ parameters for the second preset monitoring area, and perform video images with the large field of view and small resolution Acquisition, real-time transmission of the collected video images to the data processing unit 303 to perform a target detection, the detection method is the same as the first preset monitoring area target detection method.

(10) By analogy, follow the steps (2) to (9) to perform target detection and/or recognition on N areas, and finally, start target detection and/or recognition from the first preset monitoring area again, and automatically loop.

During the work of the entire intelligent visual perception device, the data processing unit 303 performs real-time analysis on the brightness of the video images collected by the camera 301-1 and 301-2, and when the brightness is insufficient, it sends control commands to the lighting unit 306-1 and the lighting unit in time. 306-2. Adjust the light intensity and light field angle of the light unit so that the brightness of the video images collected by the camera 301-1 and the camera 301-2 is moderate.

Example 8:

On the basis of the foregoing embodiment, an eighth implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. The intelligent visual perception device is composed of 2 visible light cameras, 2 transmission mechanisms, 1 data processing unit, 1 communication interface unit, 1 power management unit, 2 illumination units and 1 protective housing. The shell has 4 windows and power and signal wire interfaces. The 4 windows are sealed with light-permeable materials. Among them, 2 windows are used for video image acquisition by the camera, and the other 2 windows are used for light supplementation. 2 The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera and the illumination unit through the control commands sent by the data processing unit, including the drive motor, the horizontal shaft, the vertical shaft, and the control line. The driving motor drives the camera and the light unit to rotate 0-360 degrees horizontally around the rotating shaft, and 0-180 degrees up and down.

As shown in Figure 7, the intelligent visual perception device includes a visible light camera 301-1, a visible light camera 301-2, a transmission mechanism 302-1, a transmission mechanism 302-2, a data processing unit 303, a communication interface unit 304, a power management unit 305, The light unit 306-1, the light unit 306-2 and the protective housing constitute 307.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the window is made of light-transmitting material, which transmits the video image collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell, or fix it on the outside On the stand. In this embodiment, the protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the windows are made of light-transmitting materials, which respectively transmit the video images collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell, including the On the external bracket. In this embodiment, the shell leaves window 308-1, window 308-2, window 309-1, window 309-2, and power and signal line interface board 310, window 308-1, window 308-2, window 309 -1. The windows 309-2 are all sealed with light-permeable materials. The windows 308-1 and 308-2 are used for the video image collection of the camera 301-1 and the camera 301-2, and the windows 309-1 and 309- 2 is used for the light transmission of the light unit 306-1 and the light unit 306-2, and supplement light for the camera 301-1 and the camera 301-2 to monitor the target.

The transmission mechanism 302-1 adjusts the left and right positions of the camera 301-1 and the vertical field of view by receiving the control commands sent by the data processing unit 303, and the transmission mechanism 302-2 adjusts the left and right positions of the camera 301-2 by receiving the control commands sent by the data processing unit 303. The position of the up and down field of view. The communication interface unit 304 mainly includes wired and wireless communication interfaces. The power management unit 305 is mainly used to supply power to the entire intelligent video perception device, and in this embodiment is a battery inside the system. The lighting units 306-1 and 306-2 mainly include light-emitting devices and light intensity adjustment units, and the light source is visible light and/or infrared light. The light unit 306-1 and the camera 301-1 are fixed together, the light unit 306-2 and the camera 301-2 are fixed together, and the transmission mechanism 302-1 jointly controls the left and right swing positions of the light unit 306-1 and the camera 301-1 , The transmission mechanism 302-2 jointly controls the left and right swing positions of the light unit 306-2 and the camera 301-2.

The system workflow is as follows:

(3) The data processing unit 303 performs a target detection of the target on the collected video image through a target detection algorithm. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, the area will be classified as a background with a high probability, and it will not be classified as a suspect Target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 303 sends a control command to the camera 301-1 and the transmission mechanism 302, and the transmission mechanism 302-1 adjusts the camera The direction of the field of view 301-1 makes the direction of the field of view of the camera 301-1 align with the second preset monitoring area, and adjust the focal length of the camera 301-1 to collect video images of the second preset monitoring area with a large field of view and a small resolution, The data processing unit 303 performs one-time target detection of the target in the second preset monitoring area, and the detection method is the same as the first-time detection method of the target in the first preset monitoring area.

(5) When a suspected target is found in a target detection, the data processing unit 303 gives the first preset monitoring area feature weight Q1, which is obtained from the number of suspected targets in the area; at the same time, the PTZ parameters are set so that the field of view is large and small. The resolution covers the second preset monitoring area, so that the camera 301-1 collects video images of the monitoring area, and transmits the collected video images to the data processing unit 303 in real time.

(7) The intelligent visual perception device adjusts the direction and focal length of the field of view of the camera 301-2, zooms the height of the suspected target to 2/3 of the height of the video image, and adjusts it to the center of the field of view, and uses the small field of view and large resolution for the suspected target The first area collects video images, and transmits the collected video images to the data processing unit 303 in real time. The data processing unit 303 performs secondary target detection of suspected targets on the collected video images through a secondary target detection algorithm. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(8) When the secondary target detection determines that the suspected target is a real target, the data processing unit 303 generates target feature information and alarm information of the real target, and at the same time, the data processing unit 303 sends control commands to the camera 301 and the transmission mechanism 302- 2. The transmission mechanism 302-2 adjusts the field of view direction of the camera 301-2, adjusts the focal length of the camera 301-2 in real time, and tracks the real target detected in real time; at the same time, the data processing unit 303 sends the target feature information and alarm information to the monitoring terminal.

(9) The camera 301-2 tracks the real target. When the real target disappears or the tracking time reaches the set value, the data processing unit 303 sends a control command to the camera 301-2 and the transmission mechanism 302-2, and the transmission mechanism 302- 2 Adjust the field of view direction of the camera 301-2 so that the field of view direction of the camera 301-2 is aimed at the next real target for tracking.

(10) When the secondary target detection determines that the suspected target is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 303 performs feature description on the false target and extracts the false target Feature description information, set the update rate for the false target feature description information, optimize the false target feedback feature information database, and further improve the accuracy of a detection algorithm.

(11) The camera 301-2 tracks the true target. When all true targets disappear or the tracking time reaches the set value, the data processing unit 303 sends a control command to the camera 301-2 and the transmission mechanism 302-2, and the transmission mechanism 302- 2 Adjust the field of view direction of the camera 301-2 so that the field of view direction of the camera 301-2 is aligned with the second area, adjust the focal length of the camera 301-2, and collect the video image of the second area with a small field of view and large resolution, and the captured video The image is transmitted to the data processing unit 303 in real time, and the data processing unit 303 performs secondary target detection of the second area target on the collected video image through the video image target detection algorithm. The detection method is the same as the first area target second detection method.

(12) By analogy, follow steps (7) to (11) to perform secondary target detection and/or recognition for W areas, and finally, start from step (2) again, starting from the first preset monitoring area to detect and / Or target recognition, automatic loop.

During the work of the entire intelligent visual perception device, the data processing unit 303 performs real-time analysis on the brightness of the video images collected by the camera 301-1 and 301-2, and when the brightness is insufficient, it sends control commands to the lighting unit 306-1 and the lighting unit in time. 306-2. Adjust the light intensity of the light unit so that the brightness of the video images collected by the camera 301-1 and the camera 301-2 is moderate.

Example 9:

On the basis of the foregoing embodiment, a ninth implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. The intelligent visual perception device is composed of 1 visible light camera, 1 near-infrared camera, 2 transmission mechanisms, 1 data processing unit, 1 communication interface unit, 1 power management unit, 2 illumination units and 1 protective housing. The shell has 4 windows and power and signal wire interfaces. The 4 windows are sealed with light-permeable materials. 2 windows are used for video image acquisition by 2 cameras, and the other 2 windows are used for light supplementation by the lighting unit. 2 The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera and the illumination unit through the control commands sent by the data processing unit, including the drive motor, the horizontal shaft, the vertical shaft, and the control line. The driving motor drives the camera and the light unit to rotate 0-360 degrees horizontally around the rotating shaft, and 0-180 degrees up and down.

The communication interface unit mainly includes wired and wireless communication interfaces, the input interface is used to receive external device signals, and its connection methods include wireless and/or wired methods; among them, wireless methods include one or more of WIFI, BT, ZIGBEE, and LORA ; Wired methods include one or more of RS485, RS422, RS232, CAN bus; the output interface is used to send the signals collected or received by the system, and its connection methods include wireless and/or wired methods; among them, the wireless method includes 2G One or more of, 3G, 4G, 5G, NB-IOT; wired mode includes one or more of LAN and optical fiber.

The data processing unit is used to analyze and process the video data collected by the camera, control the camera to adjust the focus, control the transmission mechanism to adjust the angle of the camera and/or the light unit, and/or exchange information with the cloud platform or data center, and/or Interaction information with other sensors on site and/or other related systems. The power management unit is mainly used to supply power to the entire intelligent video sensing device. The light unit mainly includes a light-emitting device, a light intensity, and light range adjustment unit. The light unit and the camera are fixed together, and the transmission mechanism jointly controls the left and right swing positions of the light unit and the camera.

As shown in Figure 7, the intelligent visual perception device includes a near-infrared camera 301-1, a visible light camera 301-2, a transmission mechanism 302-1, a transmission mechanism 302-2, a data processing unit 303, a communication interface unit 304, and a power management unit 305 , The illumination unit 306-1, the illumination unit 306-2, and the protective shell constitute 307.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the window adopts light-transmitting material to transmit the video image collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell, including fixing on the outside On the stand. In this embodiment, the shell leaves window 308-1, window 308-2, window 309-1, window 309-2, and power and signal line interface board 310, window 308-1, window 308-2, window 309 -1. The windows 309-2 are all sealed with light-permeable materials. The windows 308-1 and 308-2 are used for the video image collection of the camera 301-1 and the camera 301-2, and the windows 309-1 and 309- 2 is used for the light transmission of the light unit 306-1 and the light unit 306-2, and supplement light for the camera 301-1 and the camera 301-2 to monitor the target. The transmission mechanism 302-1 adjusts the left-right and vertical field of view position of the camera 301-1 through the control commands sent by the data processing unit 303, and the transmission mechanism 302-2 adjusts the left-right and vertical vision of the camera 301-2 through the control commands sent by the data processing unit 303. Field direction position. The communication interface unit 304 mainly includes wired and wireless communication interfaces. The power management unit 305 is mainly used to supply power to the entire intelligent video sensing device, and in this embodiment is an external solar panel. The lighting units 306-1 and 306-2 mainly include light-emitting devices and light intensity adjustment units, both of which are infrared light. The light unit 306-1 and the camera 301-1 are fixed together, the light unit 306-2 and the camera 301-2 are fixed together, and the transmission mechanism 302-1 jointly controls the left and right swing positions of the light unit 306-1 and the camera 301-1 , The transmission mechanism 302-2 jointly controls the left and right swing positions of the light unit 306-2 and the camera 301-2.

The system workflow is as follows:

(2) For the first preset monitoring area, PTZ parameters are set, the near-infrared camera 301-1 collects video images with the large field of view and small resolution, and transmits the collected video images to the data processing unit 303 in real time.

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 303 sends control commands to the infrared camera 301-1 and the transmission mechanism 302-1, and the transmission mechanism 302- 1 Adjust the direction of the infrared camera 301-1's field of view so that the direction of the infrared camera 301-1's field of view is aligned with the second preset monitoring area, adjust the focal length of the infrared camera 301-1, and collect video images in this area with a large field of view and a small resolution The data processing unit 303 performs a target detection of the target on the collected video image through the video image target detection algorithm.

(5) When a suspected target is found in a target detection, adjust the PTZ parameters: the data processing unit 303 sends a control command to the visible light camera 301-2 and the transmission mechanism 302-2, and the transmission mechanism 302-2 adjusts the field of view of the visible light camera 301-2 Direction so that the visual field of the visible light camera 301-2 is aimed at the suspected target, and the focal length of the visible light camera 301-2 is adjusted. By adjusting the PTZ parameters, the height of the suspected target is scaled to 1/3 of the height of the video image, and the video image of the suspected target is collected with the small field of view and large resolution. The data processing unit 303 performs the secondary target detection algorithm to collect the video image Perform secondary target detection. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(6) When the secondary target detection determines that the suspected target is a true target, the data processing unit 303 outputs alarm information to the monitoring terminal and generates the true target characteristic information. At the same time, the data processing unit 303 sends a control command to the visible light camera 301- 2 and the transmission mechanism 302-2, the transmission mechanism 302-2 adjusts the visual field direction of the visible light camera 301-2, adjusts the focal length of the visible light camera 301-2 in real time, and tracks the real target detected in real time; at the same time, the data processing unit 303 will Target feature information and alarm information are sent to the monitoring terminal.

Target feature information includes feature information that characterizes the specific classification of the target, including people, animals, cars and/or models, flying objects, other foreign objects that should not appear, including natural falling objects and/or diffuse objects, such as falling rocks, mudslides, and One or more of human remains, or characteristic information that characterizes the specific identity of the target, including one or more of the identity of a person, the type of animal, the license plate of a car, and the type of other foreign objects. The specific information types and contents are defined according to the specific application environment.

(7) The visible light camera 301-2 tracks the real target. When the real target disappears or the tracking time reaches the set value, the data processing unit 303 sends a control command to the visible light camera 301-2 and the transmission mechanism 302-2, the transmission mechanism 302-2 adjusts the direction of the field of view of the visible light camera 301-2, so that the direction of the field of view of the visible light camera 301-2 is aligned with the next real target for tracking.

(8) When the secondary target detection determines that the suspected target is a false target, the false target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 303 performs feature description on the false target and extracts the false target Feature description information, update the false target feedback feature information database, in order to reduce the probability of a target detection error detection algorithm after this. As the running time of the system increases, the false target feature description becomes more and more accurate, the false detection probability of another target detection by the system will be lower and lower, the accuracy will be higher and higher, and the system performance will be automatically improved.

(9) The visible light camera 301-2 tracks the real target. When all real targets disappear or the tracking time reaches the set value, set the PTZ parameters for the second preset monitoring area, and perform video with the large field of view and small resolution Image acquisition, real-time transmission of the collected video images to the data processing unit 303: The data processing unit 303 sends control commands to the infrared camera 301-1 and the transmission mechanism 302-1, and the transmission mechanism 302-1 adjusts the direction of the infrared camera 301-1 field of view , The direction of the infrared camera 301-1 field of view is aligned with the second preset monitoring area, the focal length of the infrared camera 301-1 is adjusted, the video image is collected in this area, and the collected video image is transmitted to the data processing unit 303 in real time, and the data is processed The unit 303 performs a target detection of the target on the collected video image through the video image target detection algorithm, and the detection method is the same as the target detection method of the first preset monitoring area.

(10) By analogy, follow the steps (2) to (9) to perform target recognition on N areas, and finally, start target recognition from the first monitoring area again, and loop automatically.

During the work of the entire intelligent visual perception device, the data processing unit 303 analyzes the brightness of the video images collected by the infrared camera 301-1 and the visible light camera 301-2 in real time, and when the brightness is insufficient, it sends control commands to the lighting unit 306-1 and 306-1 in time. The light unit 306-2 adjusts the light intensity and light range of the light unit so that the brightness of the video images collected by the infrared camera 301-1 and the visible light camera 301-2 is moderate.

Example 10:

On the basis of the foregoing embodiment, a tenth implementation manner of an intelligent visual perception system is also provided:

As shown in Figure 7, the intelligent visual perception device includes a near-infrared camera 301-1, a visible light camera 301-2, a transmission mechanism 302-1, a transmission mechanism 302-2, a data processing unit 303, a communication interface unit 304, and a power management unit 305 , The illumination unit 306-1, the illumination unit 306-2 and the protective shell constitute 307.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the windows are made of light-transmitting materials, which respectively transmit the video images collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell and is fixed on the outside On the stand. In this embodiment, the shell leaves window 308-1, window 308-2, window 309-1, window 309-2, and power and signal line interface board 310, window 308-1, window 308-2, window 309 -1. The windows 309-2 are all sealed with light-permeable materials. The windows 308-1 and 308-2 are used for the video image collection of the camera 301-1 and the camera 301-2, and the windows 309-1 and 309- 2 is used for the light transmission of the light unit 306-1 and the light unit 306-2, and supplement light for the camera 301-1 and the camera 301-2 to monitor the target. The transmission mechanism 302-1 adjusts the left-right and vertical field of view position of the camera 301-1 through the control commands sent by the data processing unit 303, and the transmission mechanism 302-2 adjusts the left-right and vertical vision of the camera 301-2 through the control commands sent by the data processing unit 303. Field direction position. The communication interface unit 304 mainly includes wired and wireless communication interfaces. The power management unit 305 is mainly used to supply power to the entire intelligent video sensing device, which is a wired power supply in this embodiment. The lighting units 306-1 and 306-2 mainly include light-emitting devices and light intensity adjustment units, and the light source is visible light and/or infrared light. The light unit 306-1 and the camera 301-1 are fixed together, the light unit 306-2 and the camera 301-2 are fixed together, and the transmission mechanism 302-1 jointly controls the left and right swing positions of the light unit 306-1 and the camera 301-1 , The transmission mechanism 302-2 jointly controls the left and right swing positions of the light unit 306-2 and the camera 301-2.

The system workflow is as follows:

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 303 sends a control command to the infrared camera 301-1 and the transmission mechanism 302, and the transmission mechanism 302-1 is adjusted The direction of the field of view of the near-infrared camera 301-1 is such that the direction of the field of view of the near-infrared camera 301-1 is aligned with the second preset monitoring area, and the focal length of the near-infrared camera 301-1 is adjusted to monitor the second preset with a large field of view and a small resolution The video image is collected in the area, and the collected video images are transmitted to the data processing unit 303 in real time. The data processing unit 303 performs a target detection of the target in the second preset monitoring area. The detection method is the same as the first detection method of the target in the first preset monitoring area. .

(5) When a suspected target is found in a target detection, the data processing unit 303 gives the first preset monitoring area feature weight Q1, which is weighted by the number of suspected targets in the area and the moving speed of the moving target; set the PTZ parameter , Cover the second preset monitoring area with a large field of view and small resolution, the near-infrared camera 301-1 collects video images of this area, and transmits the collected video images to the data processing unit 303 in real time.

In this embodiment, each area feature weight Qi (0<i≤N) is respectively weighted by the number of suspected targets in each area and the moving speed of the moving target.

(7) The intelligent visual perception device adjusts the direction and focal length of the field of view of the near-infrared camera 301-1. By adjusting the PTZ parameters, the height of the suspected target is zoomed to 1/2 of the height of the field of view, and adjusted to the center of the field of view, so that the small field of view can be used for large resolution Rate the video image collection of the first area where there are suspected targets, and transmit the collected video images to the data processing unit 303 in real time. The data processing unit 303 performs secondary target detection algorithms on the collected video images sequentially. Target Detection. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(8) When the secondary target detection determines that the suspected target is a real target, the data processing unit 303 generates target feature information 1 and alarm information 1 of the real target, and at the same time, the data processing unit 303 sends control commands to the camera 301 and the transmission mechanism 302-2, the transmission mechanism 302-2 adjusts the visual field direction of the visible light camera 301-2, adjusts the focal length of the visible light camera 301-2 in real time, and tracks the real target detected in real time; at the same time, the data processing unit 303 detects the real target twice The target characteristic information 1 of the target is sent to the cloud platform or data center; the cloud platform or data center uses the target recognition algorithm to identify the target according to the target characteristic information 1 sent by the intelligent visual perception device, and obtains the target characteristic information 2 and alarm information 2 Send to the monitoring terminal; the monitoring terminal processes and displays the alarm information 1, the alarm information 2, the target characteristic information 1 and/or the target characteristic information 2.

(9) The visible light camera 301-2 tracks the true target. When the true target disappears or the tracking time reaches the set value, the data processing unit 303 sends a control command to the camera 301-2 and the transmission mechanism 302-2, and the transmission mechanism 302 -2 Adjust the direction of the field of view of the visible light camera 301-2 so that the direction of the field of view of the visible light camera 301-2 is aimed at the next real target for tracking.

(11) The visible light camera 301-2 tracks the real target. When all real targets disappear or the tracking time reaches the set value, the data processing unit 303 sends a control command to the visible light camera 301-2 and the transmission mechanism 302-2, the transmission mechanism 302-2 Adjust the field of view direction of the visible light camera 301-2 so that the field of view direction of the visible light camera 301-2 is aligned with the second area, adjust the focal length of the visible light camera 301-2, and collect video images of the second area with a small field of view and large resolution , The collected video images are transmitted to the data processing unit 303 in real time. The data processing unit 303 performs the second target detection of the second area target on the collected video image through the video image target detection algorithm. The detection method is the same as the second target detection method of the first area target. The detection method is the same.

(12) By analogy, follow the steps (7) to (11) to perform secondary target detection and recognition for W areas, and finally, start from step (1) again, start target recognition from the first preset monitoring area, and automatically cycle.

During the work of the entire intelligent visual perception device, the data processing unit 303 performs real-time analysis on the brightness of the video images collected by the camera 301-1 and the visible light camera 301-2, and when the brightness is insufficient, it sends control commands to the lighting unit 306-1 and lighting in time. The unit 306-2 adjusts the light intensity of the light unit so that the brightness of the video image collected by the camera 301 is moderate.

Example 11:

On the basis of the foregoing embodiment, an eleventh implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. The intelligent visual perception device is composed of 1 visible light camera, 1 infrared thermal imaging camera, 2 transmission mechanisms, 1 data processing unit, 1 communication interface unit, 1 power management unit, 1 lighting unit and 1 protective housing. The shell has 3 windows and power and signal line interfaces. The 3 windows are sealed with light-permeable materials. Among them, 2 windows are used for video image collection by 2 cameras, and the other window is used for light supplementation. 2 The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera and the illumination unit through the control commands sent by the data processing unit, including the drive motor, the horizontal shaft, the vertical shaft, and the control line. The driving motor drives the camera and the light unit to rotate 0-360 degrees horizontally around the rotating shaft, and 0-180 degrees up and down.

As shown in Figure 8, the intelligent visual perception device includes an infrared thermal imaging camera 401-1, a visible light camera 401-2, a transmission mechanism 402-1, a transmission mechanism 402-2, a data processing unit 403, a communication interface unit 404, and a power management unit 405. Illumination unit 406, and protective housing 407.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the window adopts light-transmitting material, which transmits the video image collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell and is fixed to the external bracket superior. In this embodiment, the shell is left with windows 408-1, 408-2, and 409, and a power and signal line interface board 410. The windows 408-1, 408-2, and 409 are all made of light-permeable materials. The window 408-1 is used for the video image collection of the infrared thermal imaging camera 401-1, the window 408-2 is used for the video image collection of the visible light camera 401-2, and the window 409 is used for the light unit 406 to transmit light to the visible light camera 401 -2 The monitoring target fills in light.

The transmission mechanism 402-1 adjusts the position of the infrared thermal imaging camera 401-1 in the left-right and vertical field of view by receiving the control commands sent by the data processing unit 403; the transmission mechanism 402-2 adjusts the camera 401 by receiving the control commands sent by the data processing unit 403 -2 The position of the left and right field of view and up and down. The communication interface unit 404 mainly includes wired and wireless communication interfaces. The power management unit 405 is mainly used to supply power to the entire intelligent video perception device, which is a wired power supply in this embodiment. The light unit 406 mainly includes a light emitting device and a light intensity adjustment unit, and the light source is a laser. The light unit 406 and the camera 401-2 are fixed together, the transmission mechanism 402-1 controls the left and right swinging positions of the infrared thermal imaging camera 401-1, and the transmission mechanism 402-2 jointly controls the light unit 406 and the camera 401-2 to swing left and right. Location.

The system workflow is as follows:

(2) For the first preset monitoring area, PTZ parameters are set, the infrared thermal imaging camera 401-1 collects video images with the large field of view and small resolution, and transmits the collected video images to the data processing unit 403 in real time.

(3) The data processing unit 403 performs a target detection of the target on the collected video image through a target detection algorithm. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, the area will be classified as a background with a high probability, and it will not be classified as a suspect Target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 403 sends a control command to the infrared thermal imaging camera 401-1 and the transmission mechanism 402-1, the transmission mechanism 402-1 adjusts the field of view direction of the infrared thermal imaging camera 401-1 so that the field of view of the infrared thermal imaging camera 401-1 is aligned with the second preset monitoring area, and adjusts the focal length of the infrared thermal imaging camera 401-1 to achieve a large field of view and a small resolution Video image collection is performed on the second preset monitoring area at a high rate, and the data processing unit 403 performs a target detection of the target on the collected video image through the video image target detection algorithm.

(5) When a suspected target is found in a target detection, adjust the PTZ parameters: the data processing unit 403 sends a control command to the visible light camera 401-2 and the transmission mechanism 402-2, and the transmission mechanism 402-2 adjusts the field of view of the visible light camera 401-2 Direction so that the visual field of the visible light camera 401-2 is aimed at the suspected target, and the focal length of the visible light camera 401-2 is adjusted. By adjusting the PTZ parameters, the height of the suspected target is scaled to 1/3 of the height of the video image, and adjusted to the center of the field of view. The video image of the suspected target is collected with this small field of view and large resolution. The data processing unit 403 undergoes a secondary target detection algorithm. , Perform secondary target detection on the collected video images. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(6) When the secondary target detection determines that the suspected target is a true target, the data processing unit 303 outputs alarm information 1 to the monitoring terminal, and generates the true target characteristic information 1. The data processing unit 403 sends control commands to the visible light camera 401-2 and the transmission mechanism 402-2. The transmission mechanism 402-2 adjusts the direction of the visible light camera 401-2 field of view, adjusts the focal length of the visible light camera 401-2 in real time, and responds to the detected real target Perform real-time tracking; at the same time, the data processing unit 403 sends the true target characteristic information 1 to the cloud platform or data center, and the cloud platform or data center performs target recognition algorithm according to the true target characteristic information 1 sent by the intelligent visual perception device. Identify, generate the true target feature information 2 and alarm information 2, and send the true target feature information 2 and alarm information 2 to the monitoring terminal; the monitoring terminal responds to the alarm information 1, alarm information 2, target feature information 1 and/or target feature Information 2 is processed.

(7) The visible light camera 401-2 tracks the real target. When the real target disappears or the tracking time reaches the set value, the data processing unit 403 sends a control command to the visible light camera 401-2 and the transmission mechanism 402-2, the transmission mechanism 402-2 adjusts the field of view direction of the visible light camera 401-2 so that the field of view direction of the visible light camera 401-2 is aimed at the next real target for tracking.

(8) When the secondary target detection judges that the suspected target is a fake target, the fake target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 403 characterizes the fake target and extracts the fake target Feature description information, update the false target feedback feature information database, in order to reduce the probability of a target detection error detection algorithm after this. As the running time of the system increases, the false target feature description becomes more and more accurate, the false detection probability of another target detection by the system will be lower and lower, the accuracy will be higher and higher, and the system performance will be automatically improved.

(9) The visible light camera 401-2 tracks the real target. When all real targets disappear or the tracking time reaches the set value, set the PTZ parameters for the second preset monitoring area, and perform video with the large field of view and small resolution Image acquisition, real-time transmission of the collected video images to the data processing unit 403: The data processing unit 403 sends control commands to the infrared thermal imaging camera 401-1 and the transmission mechanism 402-1, and the transmission mechanism 402-1 adjusts the infrared thermal imaging camera 401- 1 Field of view direction, so that the field of view of the infrared thermal imaging camera 401-1 is aligned with the second preset monitoring area, adjust the focal length of the infrared thermal imaging camera 401-1, and video the second preset monitoring area with a large field of view and small resolution Image acquisition, real-time transmission of the collected video images to the data processing unit 403. The data processing unit 403 performs a target detection on the collected video images through the video image target detection algorithm. The detection method is the same as the first preset monitoring area target detection The method is the same.

(10) By analogy, follow the steps (2) to (9) to perform target recognition on N areas, and finally, start target recognition from the first preset monitoring area again, and loop automatically.

During the work of the entire intelligent visual perception device, the data processing unit 403 performs real-time analysis on the brightness of the video image collected by the visible light camera 401-2, and when the brightness is insufficient, it sends a control command to the lighting unit 406 in time, and the lighting unit 406 adjusts the light intensity and light. Range, so that the visible light camera 401-2 captures video images with moderate brightness.

Example 12:

On the basis of the foregoing embodiment, a twelfth implementation manner of an intelligent visual perception system is also provided:

An intelligent visual perception system includes one or more intelligent visual perception devices. The intelligent visual perception device is composed of 1 visible light camera, 1 infrared camera, 2 transmission mechanisms, 1 data processing unit, 1 communication interface unit, 1 power management unit, 1 lighting unit and 1 protective housing. The shell has 3 windows and power and signal line interfaces. The 3 windows are sealed with light-permeable materials. Among them, 2 windows are used for video image collection by 2 cameras, and the other window is used for light supplementation. 2 The transmission mechanism adjusts the horizontal and vertical field of view positions of the camera and the illumination unit through the control commands sent by the data processing unit, including the drive motor, the horizontal shaft, the vertical shaft, and the control line. The driving motor drives the camera and the light unit to rotate 0-360 degrees horizontally around the rotating shaft, and 0-180 degrees up and down.

As shown in Figure 8, the intelligent visual perception device includes an infrared camera 401-1, a visible light camera 401-2, a transmission mechanism 402-1, a transmission mechanism 402-2, a data processing unit 403, a communication interface unit 404, a power management unit 405, Illumination unit 406, protective housing 407.

The protective shell includes an interface board, a window, and a fixing seat. There are one or more interfaces on the interface board, which are connected to the external unit; the window adopts light-transmitting material, which transmits the video image collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell and is fixed to the external bracket superior. In this embodiment, the shell is left with windows 408-1, 408-2, and 409, and a power and signal line interface board 410. The windows 408-1, 408-2, and 409 are all made of light-permeable materials. The window 408-1 is used for the video image collection of the infrared camera 401-1, the window 408-2 is used for the video image collection of the visible light camera 401-2, and the window 409 is used for the light unit 406 to transmit light to the visible light camera 401-2 The monitoring target fills in light.

The transmission mechanism 402-1 adjusts the left and right positions of the infrared camera 401-1 and the vertical field of view through the control commands sent by the data processing unit 403; the transmission mechanism 402-2 adjusts the left and right positions of the camera 401-2 through the control commands sent by the data processing unit 403 And the position of the up and down field of view. The communication interface unit 404 mainly includes wired and wireless communication interfaces. The power management unit 405 is mainly used to supply power to the entire intelligent video perception device, and in this embodiment is a battery inside the system. The light unit 406 mainly includes a light emitting device and a unit for adjusting light intensity and light range, and the light source is infrared light. The light unit 406 and the camera 401-2 are fixed together, the transmission mechanism 402-1 controls the left and right swing positions of the infrared camera 401-1, and the transmission mechanism 402-2 jointly controls the left and right swing positions of the light unit 406 and the camera 401-2.

The system workflow is as follows:

(3) The data processing unit 403 performs a target detection on the collected video image through a target detection algorithm. A target detection combines the false target feedback feature library information. If the image feature of a certain area has a high degree of matching with the false target feedback feature description of the area, the area will be classified as a background with a high probability, and it will not be classified as a suspect Target. The primary target detection algorithm is a moving target detection algorithm based on a fixed background model, and/or a target classification detection algorithm that has nothing to do with the background.

(4) When a target detection is found to be undoubtedly a target, set the PTZ parameters to cover the second preset monitoring area: the data processing unit 403 sends a control command to the infrared camera 401-1 and the transmission mechanism 402, and the transmission mechanism 402-1 is adjusted The field of view direction of the infrared camera 401-1 is such that the field of view direction of the infrared camera 401-1 is aligned with the second preset monitoring area, and the focal length of the infrared camera 401-1 is adjusted to collect video images of this area with a large field of view and a small resolution. The data processing unit 403 performs a target detection of the target on the collected video image through the video image target detection algorithm. .

(5) When a suspected target is found in a target detection, the data processing unit 403 gives the first preset monitoring area feature weight Q1, which is weighted by the number of suspected targets in the area and the moving speed of the moving target; set the PTZ parameter , Cover the second preset monitoring area with a large field of view and small resolution, so that the infrared camera 401-1 collects video images in this area, and transmits the collected video images to the data processing unit 403 in real time. After the video image target detection algorithm, One target detection.

(6) By analogy, repeat the above steps (1) ~ (5), perform a target detection of N monitoring area targets, distinguish W areas where there are suspected targets, and compare the W areas according to the size of the area feature weight Qi Re-order, the largest weight is the first area, and the smallest weight is the W-th area.

(7) The intelligent visual perception device adjusts the field of view direction and focal length of the visible light camera 401-2. By adjusting the PTZ parameters, the height of the suspected target is zoomed to 1/6 of the field of view height, and adjusted to the center of the field of view, so that the small field of view has a large resolution Video image acquisition is performed on the first area where the suspected target exists, and the data processing unit 403 passes the collected video image through a secondary target detection algorithm to sequentially perform secondary target detection of the suspected target. The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with the background.

(8) When the secondary target detection determines that the suspected target is a true target, the data processing unit 403 generates target feature information 1 and warning information 1 of the true target. The data processing unit 403 sends control commands to the camera 401 and the transmission mechanism 402-2. The transmission mechanism 402-2 adjusts the direction of the visible light camera 401-2 field of view, adjusts the focal length of the visible light camera 401-2 in real time, and tracks the real target detected in real time. ; At the same time, the data processing unit 403 sends the target feature information 1 of the real target detected twice to the cloud platform or data center; the cloud platform or data center uses the target feature information 1 sent by the intelligent visual perception device to perform the target recognition algorithm Identify and send the obtained target characteristic information 2 and alarm information 2 to the monitoring terminal; the monitoring terminal processes and displays the alarm information 1, the alarm information 2, the target characteristic information 1 and/or the target characteristic information 2.

(9) The visible light camera 401-2 tracks the real target. When the real target disappears or the tracking time reaches the set value, the data processing unit 403 sends a control command to the camera 401-2 and the transmission mechanism 402-2, and the transmission mechanism 402 -2 Adjust the direction of the field of view of the visible light camera 401-2 so that the direction of the field of view of the visible light camera 401-2 is aimed at the next real target for tracking.

(10) When the secondary target detection determines that the suspected target is a fake target, the fake target determined under the large-resolution image is mapped to the small-resolution image, and the data processing unit 403 features the fake target and extracts the fake target Feature description information, set the update rate for the false target feature description information, optimize the false target feedback feature information database, and further improve the accuracy of a detection algorithm.

(11) The visible light camera 401-2 tracks the real target. When all real targets disappear or the tracking time reaches the set value, the data processing unit 403 sends a control command to the visible light camera 401-2 and the transmission mechanism 402-2, the transmission mechanism 402-2 Adjust the field of view direction of the visible light camera 401-2 so that the field of view direction of the visible light camera 401-2 is aligned with the second area, adjust the focal length of the visible light camera 401-2, and collect video images of the second area with a small field of view and large resolution The data processing unit 403 performs the secondary target detection of the second area target through the video image target detection algorithm on the collected video image, and the detection method is the same as the first area target second detection method.

(12) By analogy, follow steps (7) to (11) to perform secondary target detection and recognition for W areas, and finally, start again from step (1), start target detection from the first preset monitoring area, and automatically cycle.

During the work of the entire intelligent visual perception device, the data processing unit 403 performs real-time analysis on the brightness of the video images collected by the camera 401-1 and the camera 401-2, and when the brightness is insufficient, it sends control commands to the light unit 406-1 and the light unit in time. 406-2. Adjust the light intensity and light range of the light unit, so that the video images captured by the camera 401-1 and the camera 401-2 have moderate brightness.

Example 13:

On the basis of the above-mentioned embodiment, the intelligent visual perception system provided by the present invention is also provided. In railway applications, according to the technical solution of embodiment 12, infrared camera 401-1 is used. According to the scope of the current monitoring area, it can be calculated to cover The maximum magnification of the entire area, by setting the PTZ parameters, the monitoring area is video image collected with the large field of view and small resolution, and the collected video image is shown in FIG. 9. The data processing unit 403 is used to perform a target detection on the collected video image through a video image target detection algorithm, and a suspicious target is found.

The data processing unit 403 sends a control command to the visible light camera 401-2 and the transmission mechanism 402-2, and the transmission mechanism 402-2 adjusts the direction of the visible light camera 401-2 field of view so that the direction of the visible light camera 401-2’s field of view is aligned with the suspected target, and adjust Visible light camera 401-2 focal length. By adjusting the PTZ parameters, the height of the suspected target is scaled to 1/3 of the height of the field of view, and adjusted to the center of the field of view. The video image of the suspected target is collected with this small field of view and large resolution. The collected video image is shown in Figure 10. .

The data processing unit 403 performs secondary target detection on the collected video image through a video image target detection algorithm, and judges it as a true target. The data processing unit sends the alarm information 1, the alarm information 2, the target characteristic information 1, and/or the target characteristic information 2 to the monitoring terminal. At the same time, the visible light camera 401-2 tracks the real target, as shown in Fig. 11. Warning information includes suspicious persons breaking into the specific monitoring area of railway operation, detention time, suspicious behavior, real dangers, etc. The target characteristic information includes people and their physical characteristics, and even the identity information of the person after facial feature comparison .

The intelligent visual perception system of the present invention realizes the detection of the surrounding driving environment along the railway. When illegal personnel break in, foreign objects are thrown in, falling rocks, and slope collapse occur within the railway driving limit, intelligent capture is carried out in time , Enlargement, feature description and warning to facilitate follow-up measures to ensure driving safety. The present invention solves the current non-intelligent monitoring of the surrounding environment of the railroad tracks. On the one hand, the monitor needs to keep a close eye on the screen, not dare to slack off, and artificially judge whether there are operational obstacles in real time; and for the actual danger that may exist in the distant place, it may be due to the image If it is not clear, it is ignored, or because the image cannot be zoomed in time, misjudgment is generated, which affects the emergency response time, delays the time of accident rescue, and may cause hundreds of millions of property losses and a real danger of numerous casualties. It can be seen that the present invention has technological advancement and extremely strong social application value.

Example 14:

On the basis of the above-mentioned embodiments, an implementation of an intelligent visual perception system is also provided. Based on the above-mentioned embodiments, the intelligent visual perception device receives information that can be covered by the intelligent visual perception device through the input interface of the communication interface. Signals of other sensing or motion devices in the detection area, when other sensing or motion devices sense abnormal conditions, send alarm information to the intelligent visual perception device, and the intelligent perception device receives the alarm information sent by the sensing or motion device, Prioritize the adjustment of the camera's field of view direction and focal length, and perform target detection in the area where the sensing or action device is located.

For example, the sensing device is a vibration sensor buried in the ground. Under normal circumstances, no vehicles will pass by the area where the vibration sensor is located. The intelligent visual perception device does not monitor the area. When a vehicle passes by, the vibration sensor will The visual perception device sends a vibration signal. After the intelligent visual perception device receives the vibration signal, it can adjust the direction and focal length of the field of view, and perform target recognition on the area where the vibration sensor is located.

Another implementation of an intelligent visual perception system, based on the above embodiments, the intelligent visual perception device provides its perception data and/or information to other sensing or action devices in the monitoring area through an output interface; further, Perform data fusion and/or joint judgment with other sensing or action devices in the monitoring area, and/or directly control or system linkage to other sensing or action devices in the monitoring area. Other sensing or action devices in the monitoring area include alarm sound and light equipment, access control equipment, firefighting equipment, obstacle removal equipment, animal repelling equipment, trailer equipment, cleaning equipment, patrol equipment, impulse mitigation equipment, emergency stop equipment, and shunt equipment One or more of, on-site and/or external communication equipment, explosion-proof equipment, medical rescue equipment, masking equipment, drone transportation equipment, personnel evacuation and/or safe evacuation equipment; the information includes warning information and/or target characteristics information.

For example, in Example 13, the intelligent visual perception system of the present invention found that the true target was a suspicious person who appeared on the railway for unknown reasons, and immediately sent perception data and information to the sensing or action device in the monitoring area, sensing or Action devices such as patrol vehicles, on-site communication equipment, etc., are connected in parallel to broadcast warning information and train arrival information. For example, through feature recognition, the system finds that the true target is a flock, etc., it can link the animal driving equipment to keep the flock away from the train track and avoid major accidents. At the same time, according to the alarm information and target feature information issued by the intelligent visual perception system of the present invention, the personnel handling process is initiated, and the relevant responsible personnel are sent to the real target area to direct and control suspicious personnel and remove obstacles on the spot to ensure the normal operation of the railway. Personnel safety.

In the above-mentioned embodiments, it may be implemented in whole or in part by any combination of software and hardware. For example, the camera inside the intelligent visual perception device has a small rotatable field of view. When other areas are required for monitoring, the housing of the intelligent visual perception device is fixed on the turntable, and the data processing unit of the intelligent visual perception device passes through the control line. Connected with the transmission mechanism in the turntable, the intelligent visual perception device controls the turntable to rotate at a large angle, and then realizes the monitoring of other areas. For example, when there are multiple cameras in the intelligent visual perception device, multiple cameras can be used to alternate the primary target detection of the target and the secondary target detection of the target to improve the recognition efficiency. When there are many targets in the monitoring area for real-time monitoring, the video image data can be uploaded to the server in the edge cloud in real time through 5G technology, as shown in Figure 4, the target is identified twice and/or three times in the service. For example, the intelligent visual perception system of the present invention includes a plurality of intelligent visual perception devices, which respectively cover different monitoring ranges to form a monitoring network, thereby expanding the monitoring field and truly realizing the overall monitoring of the protection area.

Cloud platform and/or data center, including server and software for image recognition through image target detection algorithm; monitoring terminal, one or more, used to display target recognition result information, receive alarm information, perform remote configuration and control, Including smart terminal equipment and its running management software. The smart terminal equipment includes but is not limited to one or more of a computer and a handheld. The server may be a virtual server, including one or more of a local server, an edge cloud, and a public cloud.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention. It should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

An intelligent visual perception system, characterized in that the system includes a camera with a variable focal length and a false target feedback feature information database in a designated monitoring area;

The camera collects video images of the monitored area at the first resolution, and uses a target detection algorithm to perform a target detection in the area on the collected video images, including: judging the designated based on the false target feedback feature information database Whether the monitoring area contains suspected targets;

When it is found that the monitoring area contains a suspected target, the camera is adjusted to collect a video image of the suspected target at the second resolution, and based on the collected video image, a secondary target detection algorithm is used for secondary target detection To determine whether the suspected target is a true target or a fake target;

When the suspected target is a false target, update the false target feedback feature information database according to the information of the false target;

When the suspected target is a true target, tracking and monitoring the true target;

The second resolution is greater than the first resolution;

The false target feedback feature information database includes the location information of the false target and the false target feature description information;

The video image collection of the monitoring area by the camera at the first resolution includes: the video image collection of the monitoring area by the camera with a large field of view and a small resolution;

The one-time target detection includes: based on the false target feedback feature information database, judging whether the image feature in the monitoring area matches the false target feature description information and position information of the area meets a specified condition, if it is satisfied, it is not judged as Suspected target; the primary target detection algorithm includes a moving target detection algorithm based on a fixed background model and a target classification detection algorithm that has nothing to do with the background;

When it is found that the monitoring area contains one or more suspected targets, adjust the field of view direction and angle of view of the camera, and collect video images of the suspected targets at the second resolution, including: successively with a small field of view and a large resolution Video image collection of the suspected target;

The secondary target detection algorithm is a target classification detection algorithm that has nothing to do with background;

Updating the false target feedback feature information database according to the information of the false target includes: mapping the false target determined under the large-resolution image to the small-resolution image, acquiring the position information of the false target, and extracting the false target Target feature description information, updating the false target feedback feature information database;

When the suspected target is a true target, tracking and monitoring the true target, and outputting large-resolution video data, outputting alarm information and/or target characteristic information;

The large field of view and small resolution means that the video image covers the area that needs to be monitored by setting PTZ parameters;

The small field of view and large resolution means that by adjusting PTZ parameters, the target is scaled to 1/10 to 4/5 of the height of the video image;

An update rate is set for the false target characteristic description information, and the false target feedback characteristic information database is optimized.
The intelligent visual perception system of claim 1, wherein the small field of view and large resolution refers to adjusting the PTZ parameters to make the target zoom to 1/6 to 2/3 of the height of the video image, and Adjust to the center of the field of view.
The intelligent visual perception system according to claim 1, wherein the target feature information includes feature information that characterizes the specific classification of the target and/or feature information that characterizes the specific identity of the target; the feature information that characterizes the specific classification of the target includes One or more of humans, animals, vehicles and/or models, flying objects, and designated foreign objects, wherein the designated foreign objects include one or more of natural falling objects and/or diffuse objects, and human remains The characteristic information that characterizes the specific identity of the target includes one or more of the identity of the person, the type of animal, the license plate of the car, and the type of other foreign objects.
The intelligent visual perception system of claim 1, wherein there are N monitoring areas, where N≥1; the system monitoring process includes:

Perform a target detection in the M-th monitoring area. When the first target detection finds that there is no doubt that it is a target, adjust the field of view direction and focal length of the camera, and perform a target detection on the next monitoring area with a large field of view and a small resolution. Among them, 1≤M≤N;

The real target in the M-th monitoring area is tracked. When the real target disappears or the tracking time reaches a set value, adjust the field of view direction and angle of the camera, and use a small field of view and a large resolution on the first Tracking the next said real target in M monitoring areas;

When all the true targets in the M-th monitoring area disappear or the tracking time reaches the set value, adjust the field of view direction and focal length of the camera, and perform a target detection on the next monitoring area with the large field of view and small resolution ；

Perform target detection on the N monitoring areas in turn, cyclically.
The intelligent visual perception system of claim 1, wherein there are N monitoring areas, and N≥1;

Perform a target detection on all N monitoring areas respectively, and calculate the respective feature weights of each monitoring area for the area where a suspected target is found in a target detection; according to the size of the respective feature weights of each monitoring area, perform the monitoring of each of the monitoring areas in order according to the specified order Area for secondary target detection and tracking.
The intelligent visual perception system of claim 1, wherein the system comprises one or more intelligent visual perception devices, and the intelligent visual perception devices comprise:

The camera is used for the video image collection, and includes one or more of a focus motor, a zoom motor, a drive module, and an image signal collection and processing unit;

A transmission mechanism for adjusting the direction and size of the field of view of the camera;

The data processing unit is used to analyze and process the video data collected by the camera, control the camera to adjust the focal length, control the transmission mechanism to adjust the direction and size of the camera's field of view, and/or interact with the cloud platform or data center Carry out information exchange;

The communication interface unit is used for information interaction with the cloud platform and/or data center, and/or linkage information interaction with other on-site sensing or action devices, and/or other associated systems, including wired and/or wireless interfaces ；

The power management unit is used to supply power to all power-consuming units;

The protective shell is used to encapsulate each unit and play a protective role.
The intelligent visual perception system according to claim 6, wherein the intelligent visual perception device further comprises a light unit for supplementing light to the area monitored by the camera; the light unit is one or more , Including one or more of a visible light source and an infrared light source; the transmission mechanism adjusts the direction and size of the field of view of the illumination unit.
The intelligent visual perception system according to claim 6, wherein there are one or more cameras in the intelligent visual perception device, the cameras are visible light cameras and/or infrared cameras, and the infrared cameras include near-infrared cameras. Camera and/or infrared thermal imaging camera.
8. The intelligent visual perception system of claim 8, wherein one or more cameras complete the video image acquisition in the primary target detection and/or the video image acquisition in the secondary target detection, And/or the video image collection during the real target tracking.
The intelligent visual perception system of claim 9, wherein the infrared camera completes the video image acquisition in the primary target detection; the visible light camera completes the video image acquisition in the secondary target detection , And/or the video image acquisition during the real target tracking.
The intelligent visual perception system of claim 9, wherein the visible light camera completes the video image acquisition in the primary target detection, and/or the video image acquisition in the secondary target detection, and/ Or the video image collection during the real target tracking.
The intelligent visual perception system according to claim 7, wherein the transmission mechanism is used to adjust the horizontal and/or vertical field of view direction and size of the camera and/or the light unit, and includes a drive motor, a horizontal A rotating shaft and a vertical rotating shaft; the driving motor drives the camera and/or the light unit to rotate horizontally around the rotating shaft by 0-360 degrees, and up and down by 0-180 degrees.
The intelligent visual perception system of claim 6, wherein the data processing unit adopts a chip with video image processing capabilities, integrates an image target detection algorithm program, and performs real-time video image processing; when the suspect is identified When the target is a false target, the false target is characterized, and the characteristic is fed back to the image target detection algorithm program.
The intelligent visual perception system according to claim 6, wherein the communication interface unit includes an input interface and an output interface; the input interface is used to receive signals from external devices; the output interface is used to transmit or The connection mode of the received signal includes wireless and/or wired mode; wherein, the wireless mode includes one or more of WIFI, BT, ZIGBEE, LORA, 2G, 3G, 4G, 5G, and NB-IOT; The wired mode includes one or more of AI/AO, DI/DO, RS485, RS422, RS232, CAN bus, LAN, and optical fiber.
The intelligent visual perception system of claim 14, wherein the intelligent visual perception device receives signals from other sensing or action devices in the monitoring area through the input interface, and when the sensing or When the action device sends an abnormal situation signal and/or alarm information, the intelligent visual perception device adjusts the direction and focal length of the camera's field of view, and prioritizes target detection in the area where the sensing or action device is located.
The intelligent visual perception system of claim 14, wherein the intelligent visual perception device provides its perception data and/or result information to other sensing or action devices in the monitoring area through the output interface , Used for data fusion and/or joint judgment, and/or direct control or linkage of the sensing or action device.
The intelligent visual perception system according to claim 15 or 16, wherein other sensing or action devices in the monitoring area include alarm sound and light equipment, access control equipment, fire fighting equipment, obstacle removal equipment, animal repelling equipment, Trailer equipment, cleaning equipment, inspection equipment, impulse mitigation equipment, emergency stop equipment, shunt equipment, on-site and/or external communication equipment, explosion-proof equipment, medical rescue equipment, shelter equipment, drone transportation equipment, personnel evacuation and/or One or more of the safe evacuation equipment; the information includes alarm information and/or target characteristic information.
The intelligent visual perception system of claim 6, wherein the power management unit includes a battery integrated inside the system, and/or an external solar panel, and/or a wired power supply.
The intelligent visual perception system according to claim 7, wherein the protective housing includes an interface board, a window, and a fixing seat; the interface board has one or more interfaces, which are connected to an external unit; the window The light-transmitting material is used to respectively transmit the video images collected by the camera and/or the light emitted by the light unit; the fixing seat is used to fix the protective shell on the external support.
The intelligent visual perception system of claim 1, wherein the alarm information and/or target characteristic information is output to a monitoring terminal, and/or the alarm information and/or target characteristic information is output to a data center and/or cloud The platform starts the alarm processing service, distributes information to the monitoring terminal, and completes one or more functions including alarm handling and/or intervention and/or system linkage.
The intelligent visual perception system of claim 20, wherein the cloud platform and/or data center includes a server and software for image recognition through image target detection algorithms; the monitoring terminal is one or more , Used to display the result information of target recognition, receive alarm information, perform remote configuration and control, including intelligent terminal equipment and its running management software, said intelligent terminal equipment including but not limited to one or more of computers and handhelds kind.
The intelligent visual perception system of claim 21, wherein the server comprises a virtual server, including one or more of a local server, an edge cloud, and a public cloud.