CN114120218A - River course floater monitoring method based on edge calculation - Google Patents
River course floater monitoring method based on edge calculation Download PDFInfo
- Publication number
- CN114120218A CN114120218A CN202111280254.XA CN202111280254A CN114120218A CN 114120218 A CN114120218 A CN 114120218A CN 202111280254 A CN202111280254 A CN 202111280254A CN 114120218 A CN114120218 A CN 114120218A
- Authority
- CN
- China
- Prior art keywords
- image
- detection
- model
- key frame
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000012544 monitoring process Methods 0.000 title claims abstract description 16
- 238000004364 calculation method Methods 0.000 title description 13
- 238000001514 detection method Methods 0.000 claims abstract description 46
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 abstract description 23
- 238000012549 training Methods 0.000 abstract description 10
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 238000013138 pruning Methods 0.000 abstract description 8
- 238000003062 neural network model Methods 0.000 abstract description 5
- 208000034699 Vitreous floaters Diseases 0.000 description 23
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 14
- 238000001914 filtration Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了一种基于边缘计算的河道漂浮物监测方法，包括以下步骤：步骤一、通过调取摄像头获取实时视频数据，并提取关键帧图像；步骤二、对步骤一提取的关键帧图像进行图像预处理操作，首先对提取的关键帧图像采用改进的中值滤波做平滑去噪处理，去噪的同时又能保护图像中目标边缘信息，然后对去噪后图像进行拉普拉斯锐化处理，提高图像对比度，增强图像中目标边缘信息。本发明采用选取检测速度和精度有较好平衡的神经网络模型，经过大量数据集训练之后，通过模型剪枝，在保证检测精度的同时，压缩模型大小，提高检测速度，再结合Soc芯片部署到边缘设备中，极大的降低网络中大量数据传输造成的时延，满足实际应用中的实时性需求。
The invention discloses a method for monitoring river course floating objects based on edge computing, comprising the following steps: step 1, acquiring real-time video data by calling a camera, and extracting key frame images; Image preprocessing operation, firstly, the extracted key frame image is smoothed and denoised by the improved median filter, which can protect the target edge information in the image while denoising, and then the denoised image is subjected to Laplacian sharpening. Processing to improve image contrast and enhance target edge information in the image. The present invention selects a neural network model with a good balance between detection speed and accuracy. After training with a large number of data sets, through model pruning, while ensuring detection accuracy, the size of the model is compressed and the detection speed is improved. In edge devices, the delay caused by a large amount of data transmission in the network is greatly reduced, and the real-time requirements in practical applications are met.
Description
Technical Field
The invention belongs to the technical field of river channel environment monitoring, and particularly relates to a river channel floater monitoring method based on edge calculation.
Background
The water surface floater Detection (Floating object Detection on water surface) can extract and identify Floating objects on the water surface through a Target Detection (Target Detection) technology, and at present, most of the water surface floater Detection (Target Detection) technologies adopt that a monitoring camera is arranged near the water surface to acquire water surface video data and transmit the water surface video data to a background server or a cloud for Target Detection technical processing. With the continuous development of monitoring technology, the image quality and resolution of monitoring video data are enhanced, and the volume of video data acquired at the front end is also continuously expanded. In this case, the front-end acquisition and back-end processing method may bring a great pressure to the network bandwidth, and the transmission processing of the massive data may cause a great delay in the whole process. The problems can be well solved through Edge Computing (Edge Computing), how to select a System-on-a-Chip (SOC) Chip and a target detection model which are suitable for Edge Computing and deploy the SOC Chip and the target detection model to a network camera, and the aim of the invention is to realize real-time identification and monitoring of river course floaters by carrying out Edge Computing at a data acquisition end and meet the requirements on real-time performance and detection precision in practical application.
In the prior art, more consideration is given to detection processing of acquired data, less consideration is given to the problem of network load in the data transmission process and the problem of real-time performance caused by transmission delay, and in order to solve the development condition of the prior art, the prior patents are searched, compared and analyzed, and the following technical information with high relevance to the invention is screened out:
patent scheme 1: CN202110385058.2 is suitable for a method, a system, equipment and a medium for identifying floating objects in a river channel;
the invention provides a floater identification method suitable for a river channel, which can effectively identify a to-be-detected area under a complex background. Firstly, acquiring data of an area to be detected, and preprocessing an acquired image; then, performing semantic division on the preprocessed image by using a recognition algorithm of a growing region to determine the outline of the region to be monitored; and finally, identifying the floating objects in the region to be monitored by using the neural network trained by the standard data set, and classifying.
Patent scheme 2: CN201811184499.0 is an intelligent water surface floater salvage system based on neural network and image recognition;
the invention discloses an intelligent water surface floater fishing system based on a neural network and image recognition. The method comprises the steps that firstly, water surface image data are acquired at intervals through an image acquisition and processing module, and image preprocessing operation is carried out; then, transmitting the data to a background, constructing a positive and negative sample data set, and training the positive and negative sample data set as training data in a neural network to obtain an image recognition model; and finally, operating an image recognition module at the PC end, performing image recognition, sending a direction signal through a serial port, and receiving and realizing the final fishing work by the intelligent water surface floater fishing module.
Patent scheme 3: CN 202110393368.9A method and device for identifying river course floating objects patrolled by unmanned aerial vehicles;
the invention provides a method and a device for identifying river floaters during unmanned aerial vehicle inspection, which improve the inspection capability of a river and reduce the inspection cost of the river. Firstly, carrying out image acquisition on a riverway to be detected through an unmanned aerial vehicle, then extracting a hue layer from the image and carrying out binarization processing; then determining whether the river channel to be detected has the floating objects or not according to the binary data, and counting coordinate position information, size and shape information of the floating objects; and finally, feeding back the information to a manager for timely processing.
The defects of patent scheme 1: the method mainly aims at the aspect of algorithm, the outline of the region to be monitored can be well determined by preprocessing the acquired image and performing semantic division by using a growing region algorithm, then floaters in the monitored region are classified by using a neural network, and finally effective identification of the floaters in the region to be monitored under the complex background is realized. The preprocessing of the image adopts mean filtering to eliminate noise, color space conversion to eliminate reflection, and the sobel operator is utilized to extract edge information, so that the subsequent image recognition precision can be effectively improved. However, the region growing algorithm adopted by the invention is an iterative method, the space and time overhead is large, the shadow effect processing by the region growing algorithm cannot achieve a good effect, and the requirement on the background data processing capability is high in practical application.
The defects of patent scheme 2: the invention relates to a water surface floater fishing system which integrates floater image acquisition, floater image identification, ship body control and fishing. The method mainly comprises the steps of dividing the images into different processing modules according to different functions, denoising in the aspects of image acquisition and recognition, removing illumination through a Gamma correction algorithm, taking the processed images as training data, training by using a convolutional neural network, operating an image recognition module at a PC (personal computer) end, determining the position of a floater, and transmitting position information to a salvaging module for salvaging. The whole floating object salvage system is relatively complete. However, in actual situations, as data is transmitted to the background for processing and then the moving and fishing signals are fed back, a large time delay may exist in the middle; secondly, in the invention, the collected images are used as training data, and the accuracy of identifying the floating objects by a neural network model without a larger data set training is difficult to ensure.
The defects of patent scheme 3: the invention provides a method and a device for identifying floaters in a riverway patrolled by an unmanned aerial vehicle. The method comprises the steps of reading a river channel image to be detected collected by an unmanned aerial vehicle, carrying out binarization processing on the river channel image, determining whether a floater exists according to binarization data, and feeding back information of coordinate positions and shapes of the floater to a manager. The method can effectively monitor the floating objects and has high flexibility. However, for the monitoring of the floating objects, manpower and material resources are consumed each time; secondly, for a dynamic water surface, the position of the floater is changed continuously, and for the subsequent management personnel to process, the monitored coordinate position information is not accurate; and the invention does not distinguish the type of the float, and is therefore not suitable for most surface float monitoring.
Disclosure of Invention
The invention aims to overcome the defects, and provides a river floating object monitoring method which selects a neural network model with better balance of detection speed and precision, compresses the size of the model and improves the detection speed while ensuring the detection precision through model pruning after a large number of data sets are trained, and can greatly reduce the time delay caused by the transmission of a large number of data in a network and meet the real-time requirement in practical application by combining Soc chips to be deployed in edge equipment.
In order to achieve the design purpose, the technical scheme adopted by the invention is as follows: a river course floater monitoring method based on edge calculation comprises the following steps:
acquiring real-time video data through a calling camera, and extracting a key frame image;
performing image preprocessing operation on the key frame image extracted in the step one, firstly performing smooth denoising processing on the extracted key frame image by adopting improved median filtering, and protecting target edge information in the image while denoising, and then performing Laplace sharpening processing on the denoised image, so that the image contrast is improved, the target edge information in the image is enhanced, and the identification of a following convolutional neural network is facilitated;
step three, training by adopting a latest YOLOV5 network structure through a large number of data sets to obtain a detection model with weights, pruning the model, compressing the model volume, improving the detection speed while ensuring the detection precision, and deploying the detection model into an SOC chip;
step four, the image processed in the step two is used as the input of a convolutional neural network, recognition detection is carried out through a trained network model, and a recognition result is transmitted to a background;
and fifthly, displaying on the background according to the detection result and early warning when floating objects exist.
The invention has the beneficial effects that: compared with the prior patent, the invention is mainly characterized in the following aspects: 1. massive data processing is distributed to each edge device through edge calculation, and the processing pressure of massive data in a background can be effectively relieved; 2. the image preprocessing is carried out by adopting median filtering denoising and Laplace sharpening to highlight the edge information of the target aiming at the specific environment of the water surface, so that the detection precision can be effectively improved; 3. in the mode of adopting edge calculation, a Yolov5 network model is adopted, and the balance among model volume, detection precision and detection speed is considered more. The invention adopts different data processing modes compared with other schemes, considers more the problem of network load, distributes a large amount of calculation to each edge device, can effectively relieve the calculation pressure, ensures the real-time performance of detection, and also considers the problem of edge calculation adaptation of a neural network model, because the calculation power of the edge device has certain limitation, the selected YOLOV5 network model is improved on the basis of the previous version, can well balance the calculation speed and the detection precision, and can better adapt the edge device by quantitatively pruning the YOLOV5, thereby improving the overall stability. The invention adopts the neural network model with better balance of detection speed and precision, after a large number of data sets are trained, the model is pruned, the detection precision is ensured, the size of the model is compressed, the detection speed is improved, and the model is combined with the Soc chip to be deployed in the edge equipment, so that the time delay caused by the transmission of a large number of data in the network is greatly reduced, and the real-time requirement in practical application is met.
Drawings
Fig. 1 is a schematic flow chart of a river course floater monitoring method based on edge calculation according to the invention;
FIG. 2 is a schematic diagram of a data transmission processing mode according to the present invention;
fig. 3 is a schematic diagram of a data transmission processing mode according to another embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. As shown in fig. 1-3: a river course floater monitoring method based on edge calculation comprises the following steps:
acquiring real-time video data through a calling camera, and extracting a key frame image;
performing image preprocessing operation on the key frame image extracted in the step one, firstly performing smooth denoising processing on the extracted key frame image by adopting improved median filtering, and protecting target edge information in the image while denoising, and then performing Laplace sharpening processing on the denoised image, so that the image contrast is improved, the target edge information in the image is enhanced, and the identification of a following convolutional neural network is facilitated;
step three, training by adopting a latest YOLOV5 network structure through a large number of data sets to obtain a detection model with weights, pruning the model, compressing the model volume, improving the detection speed while ensuring the detection precision, and deploying the detection model into an SOC chip;
step four, the image processed in the step two is used as the input of a convolutional neural network, recognition detection is carried out through a trained network model, and a recognition result is transmitted to a background;
and fifthly, displaying on the background according to the detection result and early warning when floating objects exist.
Examples are as follows:
s1: training a data set by using a YOLOV5 network model, and pruning and compressing the trained model by using a magnitude-based pruning mode, wherein a pruning algorithm is as follows:
1. for each filter Fi，jThe sum of all weights (weight) therein is denoted as Sj；
2. To the filter according to SjSorting the values of (a);
3. according to a certain proportion, removing SjA filter of relatively low value;
4. after a layer is cut separately, the accuracy rate after cutting is calculated, and for a sensitive layer, a filter is removed by adopting a smaller proportion, or the sensitive layer is skipped without cutting;
s2: and acquiring real-time video data by calling a camera, and extracting a video frame image.
S3: preprocessing the acquired image: the method comprises the following steps of carrying out denoising and Laplace sharpening on an original image by using an improved median filtering strategy, wherein the specific algorithm is as follows:
standard median filtering algorithm: and scanning the whole image by adopting a sliding window and taking each pixel point as a window center, sequencing the brightness values of the pixel points of each window, and assigning values to the center pixel points by taking the values.
On the basis, whether the central pixel point of the sliding window is the maximum value or the minimum value of the window is judged, if yes, standard median filtering processing is carried out on the pixel point, and if not, processing is not carried out;
the laplacian sharpening algorithm is as follows:
and (3) taking the value of the function of the position represented by the pixel value of the central pixel point, marking as f (x, y), detecting whether the edge exists through first-order differential, and determining the edge position through second-order differential, wherein the formula is as follows:
a four-neighborhood template matrix can be obtained:
and then calculating a pixel point replacement value:
replacing the original pixel value f (x, y) with the calculated value g (x, y),
s4: detection and identification: transmitting the preprocessed image into a trained convolutional neural network model for detection and identification;
s5: and after the front-end data is processed, the detection result is returned to the background to display the floating object condition in the detection area and give timely early warning.
Claims (1)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111280254.XA CN114120218A (en) | 2021-10-29 | 2021-10-29 | River course floater monitoring method based on edge calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111280254.XA CN114120218A (en) | 2021-10-29 | 2021-10-29 | River course floater monitoring method based on edge calculation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114120218A true CN114120218A (en) | 2022-03-01 |
Family
ID=80380083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111280254.XA Pending CN114120218A (en) | 2021-10-29 | 2021-10-29 | River course floater monitoring method based on edge calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114120218A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115249339A (en) * | 2022-06-10 | 2022-10-28 | 广州中科云图智能科技有限公司 | River floating object identification system, method, equipment and storage medium |
| CN116385530A (en) * | 2023-06-01 | 2023-07-04 | 太湖流域水文水资源监测中心（太湖流域水环境监测中心） | River and lake floater target identification method based on Internet of things technology |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108647648A (en) * | 2018-05-14 | 2018-10-12 | 电子科技大学 | A kind of Ship Recognition system and method under visible light conditions based on convolutional neural networks |
| CN111024158A (en) * | 2019-12-23 | 2020-04-17 | 广东工业大学 | An intelligent monitoring method for kitchen appliance hazards combined with edge computing |
| CN113011588A (en) * | 2021-04-21 | 2021-06-22 | 华侨大学 | Pruning method, device, equipment and medium for convolutional neural network |
-
2021
- 2021-10-29 CN CN202111280254.XA patent/CN114120218A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108647648A (en) * | 2018-05-14 | 2018-10-12 | 电子科技大学 | A kind of Ship Recognition system and method under visible light conditions based on convolutional neural networks |
| CN111024158A (en) * | 2019-12-23 | 2020-04-17 | 广东工业大学 | An intelligent monitoring method for kitchen appliance hazards combined with edge computing |
| CN113011588A (en) * | 2021-04-21 | 2021-06-22 | 华侨大学 | Pruning method, device, equipment and medium for convolutional neural network |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115249339A (en) * | 2022-06-10 | 2022-10-28 | 广州中科云图智能科技有限公司 | River floating object identification system, method, equipment and storage medium |
| CN115249339B (en) * | 2022-06-10 | 2024-05-28 | 广州中科云图智能科技有限公司 | River float recognition system, method, equipment and storage medium |
| CN116385530A (en) * | 2023-06-01 | 2023-07-04 | 太湖流域水文水资源监测中心（太湖流域水环境监测中心） | River and lake floater target identification method based on Internet of things technology |
| CN116385530B (en) * | 2023-06-01 | 2023-08-08 | 太湖流域水文水资源监测中心（太湖流域水环境监测中心） | River and lake floater target identification method based on Internet of things technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106651872B (en) | Pavement crack identification method and system based on Prewitt operator | |
| CN108564085B (en) | A method of automatically reading the reading of a pointer-type meter | |
| CN109472200B (en) | Intelligent sea surface rubbish detection method, system and storage medium | |
| CN109886170B (en) | A kind of oncomelania intelligent detection identification and statistics system | |
| CN113324864A (en) | Pantograph carbon slide plate abrasion detection method based on deep learning target detection | |
| CN116758045B (en) | Surface defect detection method and system for semiconductor light-emitting diode | |
| CN112132166A (en) | Intelligent analysis method, system and device for digital cytopathology image | |
| CN114120218A (en) | River course floater monitoring method based on edge calculation | |
| CN105023272A (en) | Crop leaf insect pest detection method and system | |
| CN115147746B (en) | Saline-alkali geological identification method based on unmanned aerial vehicle remote sensing image | |
| CN106815819A (en) | Many strategy grain worm visible detection methods | |
| CN112418028A (en) | Satellite image ship identification and segmentation method based on deep learning | |
| CN110047079A (en) | A kind of optimum segmentation scale selection method based on objects similarity | |
| CN109815974A (en) | A kind of cell pathology slide classification method, system, equipment, storage medium | |
| CN114029943A (en) | Target grabbing and positioning method and system based on image data processing | |
| CN111126162A (en) | Method, device and storage medium for identifying inflammatory cells in image | |
| CN108765439A (en) | A kind of sea horizon detection method based on unmanned water surface ship | |
| CN107992799B (en) | Preprocessing methods for smoke detection applications | |
| CN118053076B (en) | Algae identification method, medium and system based on high-resolution images | |
| CN113052234A (en) | Jade classification method based on image features and deep learning technology | |
| CN110276260B (en) | A product detection method based on depth camera | |
| CN117541832A (en) | Abnormality detection method, abnormality detection system, electronic device, and storage medium | |
| CN112907651B (en) | Measuring method of oyster external form based on semantic segmentation network | |
| CN112396031B (en) | Target detection method and system based on heterogeneous operation platform | |
| CN114881984A (en) | Detection method and device for rice processing precision, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |